Title: Practical $0.385$-Approximation for Submodular Maximization Subject to a Cardinality Constraint
Venue: NeurIPS 2024
Accepted: yes
Keywords: Submodular maximization

Discrete optimization

Machine learning
Abstract: Non-monotone constrained submodular maximization plays a crucial role in various machine learning applications. However, existing algorithms often struggle with a trade-off between approximation guarantees and practical efficiency. The current state-of-the-art is a recent $0.401$-approximation algorithm, but its computational complexity makes it highly impractical. The best practical algorithms for the problem only guarantee $1/e$-approximation. In this work, we present a novel algorithm for submodular maximization subject to a cardinality constraint that combines a guarantee of $0.385$-approximation with a low and practical query complexity of $O(n+k^2)$. Furthermore, we evaluate our algorithm's performance through extensive machine learning applications, including Movie Recommendation, Image Summarization, and more. These evaluations demonstrate the efficacy of our approach.

Practical 0 . 385 -Approximation for Submodular Maximization Subject to a Cardinality Constraint

Abstract

Non-monotone constrained submodular maximization plays a crucial role in various machine learning applications. However, existing algorithms often struggle with a trade-off between approximation guarantees and practical efficiency. The current state-of-the-art is a recent 0 . 401 -approximation algorithm, but its computational complexity makes it highly impractical. The best practical algorithms for the problem only guarantee 1 /e -approximation. In this work, we present a novel algorithm for submodular maximization subject to a cardinality constraint that combines a guarantee of 0 . 385 -approximation with a low and practical query complexity of O ( n + k 2 ) , where n is the size of the ground set and k is the maximum size of a feasible solution. Furthermore, we evaluate the empirical performance of our algorithm in experiments based on the machine learning applications of Movie Recommendation, Image Summarization, and Revenue Maximization. These experiments demonstrate the efficacy of our approach.

1 Introduction

In the last few years, the ability to effectively summarize data has gained importance due to the advent of massive datasets in many fields. Such summarization often consists of selecting a small representative subset from a large corpus of images, text, movies, etc. Without a specific structure, this task can be as challenging as finding a global minimum of a non-convex function. Fortunately, many practical machine learning problems exhibit some structure, making them suitable for optimization techniques (either exact or approximate).

A key structure present in many such problems is submodularity, also known as the principle of diminishing returns. This principle suggests that the incremental value of an element decreases as the set it is added to grows. Submodularity enables the creation of algorithms that can provide near-optimal solutions, making it fundamental in machine learning. It has been successfully applied to various tasks, such as social graph analysis [39], adversarial attacks [26, 36], dictionary learning [13], data summarization [32, 34, 35], interpreting neural networks [14], robotics [45, 41], and many more.

To exemplify the notion of submodularity, consider the following task. Given a large dataset, our goal is to identify a subset that effectively summarizes (or covers) the data, with a good representative

set being one that covers the majority of the data. Note that adding an element s to a set B is less beneficial to this goal than adding it to a subset A ⊂ B due to the higher likelihood of overlapping coverage. Formally, if N is the set of elements in the dataset, and we define a function f : 2 N → R mapping every set of elements to its coverage, then, the above discussion implies that, for every two sets A ⊆ B ⊆ N and element s ∈ N \ B , it must hold that f ( s | A ) ≥ f ( s | B ) , where f ( s | A ) ≜ f ( { s } ∪ A ) -f ( A ) denotes the marginal gain of the element s with respect to the set A . We say that a set function is submodular if it obeys this property.

Unfortunately, maximizing submodular functions is NP-hard even without a constraint [16], and therefore, works on maximization of such functions aim for approximations. Many of these works make the extra assumption that the submodular function f : 2 N → R is monotone , i.e., that for every two sets A ⊆ B ⊆ N , it holds that f ( B ) ≥ f ( A ) . Two of the first works of this kind, by Nemhauser and Wolsey [37] and Nemhauser et al. [38], showed that a greedy algorithm achieves a tight 1 -1 / e approximation for the problem of maximizing a non-negative monotone submodular function subject to a cardinality constraint using O ( nk ) function evaluations, where n is the size of the ground set N and k is the maximum cardinality allowed for the output set. An important line of work aimed to improve the time complexity of the last algorithm, culminating with deterministic and randomized algorithms that have managed to reduce the time complexity to linear at the cost of an approximation guarantee that is worse only by a factor of 1 -ε [6, 31, 27, 22, 20].

Unfortunately, the submodular functions that arise in machine learning applications are often nonmonotone, either because they are naturally non-monotone, or because a diversity-promoting nonmonotone regularizer is added to them. Maximizing a non-monotone submodular function is challenging. The only tight approximation known for such functions is for the case of unconstrained maximization, which enjoys a tight approximation ratio of 1 / 2 [16, 7]. A slightly more involved case is the problem of maximizing a non-negative (not necessarily monotone) submodular function subject to a cardinality constraint. This problem has been studied extensively. First, Lee et al. [25] suggested an algorithm guaranteeing (1 / 4 -ε ) -approximation for it. This approximation ratio was improved in a long series of works [5, 11, 15, 43], leading to a very recent 0 . 401 -approximation algorithm due to Buchbinder and Feldman [4], which improved over a previous 0 . 385 -approximation algorithm due to Buchbinder and Feldman [3]. On the inapproximability side, it has been shown that no algorithm can guarantee a better approximation ratio than 0 . 478 in polynomial time [40].

Most of the results in the above-mentioned line of work are only of theoretical interest due to a very high time complexity. The two exceptions are the Random Greedy algorithm of Buchbinder et al. [5] that guarantees 1 / e -approximation using O ( nk ) queries to the objective function, and the Sample Greedy algorithm of Buchbinder et al. [6] that reduces the query complexity to O ε ( n ) at the cost of a slightly worse approximation ratio of 1 / e -ε .

1.1 Our contribution

In this work, we introduce a novel combinatorial algorithm for maximizing a non-negative submodular function subject to a cardinality constraint. Our suggested method combines a practical query complexity of O ( n + k 2 ) with an approximation guarantee of 0 . 385 , which improves over the 1 / e -approximation of the state-of-the-art practical algorithm. To emphasize the effectiveness of our suggested method, we empirically evaluate it on 3 applications: (i) Movie Recommendation, (ii) Image Summarization, and (iii) Revenue Maximization. Our experiments on these applications demonstrate that our algorithm (Algorithm 3) outperforms the current practical state-of-the-art algorithms.

Remark. An independent work that recently appeared on arXiv [12] suggests another 0 . 385 -approximation algorithm for our problem using O ( nk ) oracle queries. Interestingly, their algorithm is very similar to a basic version of our algorithm presented in Appendix A. In this work, our main goal is to find ways to speed up this basic algorithm, which leads to our main result. In contrast, the main goal of [12] is to derandomize the basic algorithm and extend it to other constraints.

1.2 Additional notation

Let us define some additional notation used throughout the paper. Given an element u ∈ N and a set S ⊆ N , we use S + u and S -u as shorthands for S ∪ { u } and S \ { u } , respectively. Given also a set function f : 2 N → R , we recall that f ( u | S ) is used to denote the marginal contribution of u to

S . Similarly, given an additional set T ⊆ N , we define f ( T | S ) ≜ f ( S ∪ T ) -f ( S ) . Finally, we denote by OPT an arbitrary optimal solution for the problem we consider.

2 Method

In this section, we present our algorithm for non-monotone submodular maximization under cardinality constraints, which is the algorithm used to prove the main theoretical result of our work (Theorem 2.3). We begin with a brief overview of our algorithm. Motivated by the ideas underlying the impractical 0 . 385 -approximation algorithm of [3], our algorithm comprises three steps:

Initial Solution: We start by searching for a good initial solution that guarantees a constant approximation to the optimal set. This is accomplished by running the recent deterministic 1 / 4 -approximation algorithm of Balkanski et al. [2]. 1

Accelerated Local Search (Algorithm 1): Next, the algorithm aims to find an (approximate) local optimum set Z using a local search method. This can be done using a classical local search algorithm at the cost of O ε ( nk 2 ) queries (see Appendix A for more detail). As an alternative, we introduce, in Subsection 2.1, our accelerated local search algorithm FAST-LOCAL-SEARCH (Algorithm 1), which reduces the query complexity to O ε ( n + k 2 ) .

Accelerated Stochastic Greedy Improvement (Algorithm 2): It can be shown that when the set Z does not have a good value, it contains only little of the value of the optimal solution, and at the same time, it contains many of the elements that negatively affects this optimal solution. Thus, it makes sense to try to avoid this set. Accordingly, after obtaining the set Z , our algorithm constructs a second possible solution using a stochastic greedy algorithm that picks only elements of N \ Z in its first iterations. One can use for this purpose a version of the Random Greedy algorithm suggested by Buchbinder et al. [5] that uses O ( nk ) queries (see Appendix A for details). To get the same result using fewer queries, we employ Algorithm 2 (described in Subsection 2.2), which is accelerated using ideas borrowed from the Sample Greedy algorithm of [6].

Our final algorithm (given as Algorithm 3 in Subsections 2.3) returns the better among the two sets produced in the last two steps (i.e., the output sets of Algorithm 1, and Algorithm 2). Intuitively, this algorithm guarantees our target approximation ratio of 0 . 385 because when f ( Z ) is smaller than this value, the set Z is bad enough that avoiding it (in the first iterations) allows Algorithm 2 to get a good enough solution.

2.1 Fast local search

In this section, we present our accelerated local search algorithm, which is the algorithm used to implement the first two steps of our main algorithm. The properties of this algorithm are formally given by Theorem 2.1. Let OPT be an optimal solution.

Theorem 2.1. There exists an algorithm that given a positive integer k , a value ε ∈ (0 , 1) , and a non-negative submodular function f : 2 N → R ≥ 0 , outputs a set S ⊆ N of size at max k that, with probability at least 1 -ε , obeys

Furthermore, the query complexity of the above algorithm is O ε ( n + k 2 ) .

Note that the guarantee of Theorem 2.1 is similar to the guarantee of a classical local search algorithm (see Appendix A for details). However, such a classical local search algorithm uses O ε ( nk 2 ) queries, which is higher than the number of queries required for the algorithm from Theorem 2.1.

We defer the formal proof of Theorem 2.1 to Appendix B. However, we note that this proof is based on Algorithm 1. Algorithm 1 implicitly assumes that the ground set N includes at least k +1 dummy

elements that always have a zero marginal contribution to f . Such elements can always be added to the ground set (before executing the algorithm) without affecting the properties of f , and removing them from the output set of the algorithm does not affect the guarantee of Theorem 2.1.

Algorithm 1: FAST-LOCAL-SEARCH ( k, f, ε, L )

input : A positive integer k ≥ 1 , a submodular function f , an approximation factor ε ∈ (0 , 1) , and a number L of iterations.

output : A subset of N of cardinality at most k .

1 Initialize S 0 to be a feasible solution that with probability at least 1 -ε provides c -approximation for the problem for some constant c ∈ (0 , 1] .

14 return FAILURE.

Algorithm 1 starts by finding an initial solution S 0 guaranteeing constant approximation (we implement this step using the deterministic 1 / 4 -approximation algorithm of Balkanski et al. [2]). If the size of the initial solution is less than k (i.e., | S 0 | < k ), the algorithm adds to it k -| S 0 | dummy elements. Then, Algorithm 1 makes roughly log 2 ε -1 attempts to find a good output. Each attempt trys to improve the (same) initial solution using L iterations. Each iteration consisting of three steps: In Step (i), the algorithm samples n k items, and picks the element u from the sample with the largest marginal contribution to the current solution S i -1 . If there are no elements in the sample with a positive marginal contribution, the algorithm picks a dummy element outside S i -1 as u . In Step (ii), the algorithm picks the element v ∈ S i -1 that has the lowest marginal value, i.e., the element whose removal from S i -1 would lead to the smallest drop in value. In Step (iii), the algorithm swaps the elements u and v if such a swap increases the value of the current solution. Once L iterations are over, the algorithm picks a uniformly random solution among all the solutions seen during this attempt (recall that the algorithm makes roughly log 2 ε -1 attempts to find a good solution). If the random solution found obeys the technical condition given on Line 13, then the algorithm returns it. Otherwise, the algorithm continues to the next attempt. If none of the attempts returns a set, the algorithm admits failure.

2.2 Guided stochastic greedy

In this section, we prove Theorem 2.2, which provides the last step of our main algorithm.

Theorem 2.2. There exists an algorithm that given a positive integer k , a value ε ∈ (0 , 1) , a value t s ∈ [0 , 1] , a non-negative submodular function f : 2 N → R ≥ 0 , and a set Z ⊆ N obeying the inequalities stated in Theorem 2.1, outputs a solution S k , obeying

where α ≜ 1 -1 /k . Moreover, this algorithm requires only O ε ( n ) queries to the objective function.

The algorithm used to prove Theorem 2.2 is Algorithm 2. This algorithm starts with an empty set and adds elements to it in iterations (at most one element per iteration) until its final solution is ready after k iterations. In its first ⌈ k · t s ⌉ iterations, the algorithm ignores the elements of Z , and in the other iterations, it considers all elements. However, except for this difference, the behavior of the algorithm in all iterations is very similar. Specifically, in each iteration i the algorithm does the following two steps. In Step (i), the algorithm samples a subset M i containing O ε ( n/k ) elements from the data. In Step (ii), the algorithm considers a subset of M i (of size either s 1 ⌈ p ( n -| Z | ) ⌉ or s 2 ⌈ pn ⌉ ) containing the elements of M i with the largest marginal contributions with respect to the current solution S i -1 , and adds a uniformly random element out of this subset to the solution (if this element has a positive marginal contribution).

Algorithm 2: Guided Stochastic Greedy

2.3 0 . 385 -Approximation guarantee

In this section, our objective is to prove the following theorem.

Theorem 2.3. Given an integer k ≥ 1 and a non-negative submodular function f : 2 N → R ≥ 0 , there exists an 0 . 385 -approximation algorithm for the problem of finding a set S ⊆ N of size at most k maximizing f . This algorithm uses O ( n + k 2 ) queries to the objective function.

The algorithm used to prove Theorem 2.3 is Algorithm 3. Our technical guarantee for Algorithm 3 is given as Lemma 2.4. When k is large enough, this lemma immediately implies Theorem 2.3 by choosing ε to be a small enough positive constant. If k is small, getting Theorem 2.3 from Lemma 2.4 requires a three steps process. First, we choose an integer constant ρ such that ρk is large enough, and we create a new ground set N ρ = { u i | u ∈ N , i ∈ [ ρ ] } and a new objective function g : 2 N ρ → R defined as g ( S ) = E [ f ( R ( S ))] , where R ( S ) is a random subset of N that includes every element u ∈ N with probability | S ∩ ( { u } × [ ρ ]) | /ρ . Then, we use Lemma 2.4 to get a set ˆ S that provides 0 . 385 -approximation for the problem max { g ( S ) | | S | ≤ ρk } . Finally, the Pipage Rounding technique of [8] can be used to get from ˆ S a 0 . 385 -approximation for our original problem. Notice that since the size of ˆ S is constant (as we consider the case of a small k ), this rounding can be done using a constant number of queries to the objective.

Lemma 2.4. Algorithm 3 makes O ε ( n + k 2 ) queries to the objective function, and returns a set whose expected value is at least ( c -O ( ε + k -1 )) f ( OPT ) for some constant c > 0 . 385 .

input : A positive integer k ≥ 1 , a non-negative submodular function f , error parameter ε ∈ (0 , 1) , and a flip point 0 ≤ t s ≤ 1

output : A set S L ⊆ N

1 Z ← FAST-LOCAL-SEARCH ( k, f, ε, L := ⌈ 2 k/ ( ε (1 -1 / e )) ⌉ .

2 if the last algorithm did not fail then

3 A ← GUIDED-STOCHASTIC-GREEDY ( Z, k, t s , ε ) .

4 return the set maximizing f among Z and A .

5 else return ∅ .

Proof. According to the proof of Theorem 2.1, our choice of the parameter L in Algorithm 1 guarantees that with probability at least 1 -ε the set Z obeys the inequalities

Let us denote by E the event that these inequalities hold. By Theorem 2.2,

Since the output of Algorithm 2 is the better set among A and Z , we can lower bound its value by any convex combination of lower bounds on the values of A and Z . More formally, if we denote by p 1 , p 2 and p 3 any three non-negative values that add up to 1 , then we get

To simplify the above inequality, we need to bound some of the terms in it. First,

where the first inequality holds since ⌈ t s · k ⌉ ≤ t s · k +1 , α ≤ 1 , the second inequality follows since α k (1 -t s ) ≥ e t s -1 ( 1 -1 k ) 1 -t s , and the last inequality holds since e t s -1 ≤ 1 . Second,

where the first inequality holds since α k -1 ≥ α k ≥ e -1 ( 1 -1 k ) , and the second inequality holds since α k -t s · k -2 ≤ e t s -1 +4 /k . Finally, it holds that α k -α k -⌈ t s · k ⌉ ≥ e -1 (1 -1 k ) -e t s -1 / (1 -1 k ) ≥ e -1 (1 -1 k ) -e t s -1 (1 + 1 k ) ≥ -e t s -1 (1 -e -t s ) -2 k .

Plugging all the above lower bounds into Inequality (1) yields the promised simplified guarantee that

By [3], for an appropriate choice of values for p 1 , p 2 , p 3 and t s the last inequality implies

for some constant c > 0 . 385 . To get from the last inequality the bound on E [max { f ( A ) , f ( Z ) } | E ] stated in the lemma, we need to show that the conditioning on E and the last term of the inequality can both be dropped. To see why the conditioning can be dropped, note that the event E happens with probability at least 1 -ε , and when it does not happen the set returned by the algorithm still has a nonnegative value. These observations show together that removing the conditioning on E in Inequality (2) only affects the constant inside the big O notation. Notice now that since OPT ∩ Z ⊆ OPT is always a feasible solution, it deterministically holds that f ( OPT ∩ Z ) ≤ f ( OPT ) . Similarly, since Z is a feasible solution, the submodularity of f guarantees that f ( OPT ∪ Z ) + f ( OPT ∩ Z ) ≤ f ( OPT ) + f ( Z ) ≤ 2 f ( OPT ) . These two bounds allow us to drop the last term of Inequality (2) at the cost of increasing (again) the constant inside the big O notation.

To complete the proof of the lemma, note that Line 1 of Algorithm 3 requires O ε ( n + k 2 ) queries to the objective function as shown in the proof of Theorem 2.1, while Line 3 of Algorithm 3 requires O ε ( n ) queries to the objective function as dictated by Theorem 2.2.

3 Experiments

To emphasize the effectiveness of our suggested method from Section 2, in this section, we empirically compare Algorithm 3 with two benchmark algorithms on three machine-learning applications: movie recommendation, image summarization, and revenue maximization. Each one of these applications necessitates maximization of a non-monotone submodular function. The benchmark algorithms we consider are the Random Greedy algorithm of Buchbinder et al. [5], and the Random Sampling algorithm of [6]. These algorithms are the current state-of-the-art practical algorithms for maximizing non-monotone submodular functions.

As stated, Algorithm 2 requires O ( n · ln ε -1 ε 2 ) queries to the objective function, where the dependence on ε comes from the choice of value for the parameter p of the algorithm. However, we have found out that in practice a more modest choice of value for p suffices. Specifically, in our experiments, we have replaced Line 2 of Algorithm 2 with p ← min { 1 , 8 k · ε } . Throughout the experiments, we have set ε = 0 . 1 ; and all the reported results are averaged across 8 executions. We use shades in our plots to depict the standard deviations of the individual results obtained in these 8 executions.

Software/Hardware . Our algorithms were implemented in Python 3.11 using mainly 'Numpy' [19], and Numba [23]. The implementations' code can be found at https://github.com/muradtuk/ 385ApproximationSubMax . The experiments were performed on a 2 . 2 GHz i9-13980HX (24 cores total) machine with 64 GB RAM.

3.1 Personalized movie recommendation

Consider a movie recommendation system in which each user specifies what genres they are interested in, and the system has to provide a representative subset of movies from these genres. Assume that each movie is represented by a vector consisting of users' ratings for the corresponding movie. One challenge here is that each user does not necessarily rate all the movies. Hence, the vectors representing the movies do not necessarily have similar sizes. To overcome this challenge, low-rank matrix completion techniques [9] can be performed on the matrix with missing values to obtain a complete rating matrix. Formally, given a few ratings from k users to n movies we obtain in this way a rating matrix M of size k × n . Following [33, 30], to score the quality of a selected subset of

movies, we use the function f ( S ) = ∑ u ∈N ∑ v ∈ S s u,v -λ ∑ u ∈ S ∑ v ∈ S s u,v . Here, N is the set of n movies, λ ∈ [0 , 1] is a parameter and s u,v denotes the similarity between movies u and v (the similarity s u,v can be calculated based on the matrix M in multiple ways: cosine similarity, inner product, etc). Note that the first term in f 's definition captures the coverage, while the second term captures diversity. Thus, the parameter λ controls the importance of diversity in the returned subset. For any λ ≤ 0 . 5 , f ( S ) is monotone [34], however, it can be non-monotone for larger values of λ .

We followed the experimental setup of the prior works [33, 30] and used a subset of movies from the MovieLens data set [18] which includes 10 , 437 movies. Each movie in this data set is represented by a 25 dimensional feature vector calculated using user ratings, and we used the inner product similarity to obtain the similarity values s u,v based on these vectors. When experimenting with this application, we fixed λ to be either 0 . 55 or 0 . 75 , and varied k .

20

40

60

80

100

3.92k

5.88k

7.85k

9.81k

11.77k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Function value

(a) Function values for

λ

= 0

.

75

.

20

40

60

80

100

5.35k

7.14k

8.92k

10.7k

12.49k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Function value

(b) Function values for

λ

= 0

.

55

.

20

40

60

80

100

0.213M

0.425M

0.638M

0.85M

1.063M

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

(c) Number of queries for

λ

= 0

.

75

.

20

40

60

80

100

0.213M

0.425M

0.638M

0.85M

1.063M

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

(d) Number of queries for

λ

= 0

.

55

.

The results of these experiments are depicted in Figure 1. One can observe that our proposed method, Algorithm 3, demonstrates superior performance compared to the other methods. Moreover, this performance is stable, and presents a much smaller variance compared to the variance in the performance of the two benchmark algorithms. The number of queries used by our algorithm is only slightly larger than the number of queries used by the Random Sampling algorithm of [6], and is typically smaller than the number of queries used by the Random Greedy algorithm of Buchbinder et al. [5], sometimes by as much as a factor of 2 .

3.2 Personalized image summarization

Consider a setting in which we get as input a collection N of images from ℓ disjoint categories (e.g., birds, dogs, cats) and the user specifies r ∈ [ ℓ ] categories, and then demands a subset of the images in these categories that summarizes all the images of the categories. Following [30] again, to evaluate a given subset of images, we use the function f ( S ) = ∑ u ∈N max v ∈ S s u,v -1 |N| ∑ u ∈ S ∑ v ∈ S s u,v , where s u,v is a non-negative similarity between images u and v .

To obtain the similarity between pair of images u, v , we utilized the DINO-VITB16 model [10] from HuggingFace [44] as the feature encoder for vision datasets. Specifically, the final layer CLS token

embedding output was used as the feature representation. The similarity between pairs of images was then computed as the cosine similarity of the corresponding embedding vectors. To experiment in this setting, we used three datasets: (i) CIFAR10 [21] - A dataset of 50 , 000 images belonging to 10 different classes (categories). (ii) CIFAR100 [21] - A dataset of 50 , 000 images belonging to 100 different classes. (iii) Tiny ImageNet [24] - A dataset of 100 , 000 images belonging to 200 different classes. In each one of our experiments, the task was to summarize a set of 10 , 000 images sampled uniformly from one of these datasets. The upper bound k on the number images allowed in the summary varied between experiments. The results of our experiments are depicted in Figure 2.

Similarly to Section 3.1, we observe that our algorithm (Algorithm 3) produces higher values compared to the state-of-the-art practical algorithms, and enjoys a lower variance in the quality of its output. However, due to the small values used for k , our algorithm requires significantly more queries to the objective function compared to the two other algorithms.

2

4

6

8

10

8250

8360

8470

8580

8690

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

Function value

k

(a) Function values for the CIFAR10 dataset.

2

4

6

8

10

9120

9240

9360

9480

9600

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Function value

(c) Function values for the Tiny ImageNet dataset.

2

4

6

8

10

27.6k

55.2k

82.7k

110.3k

137.9k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

(e) Number of queries for the CIFAR100 dataset.

2

4

6

8

10

8280

8400

8520

8640

8760

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

Function value

k

(b) Function values for the CIFAR100 dataset.

2

4

6

8

10

27.6k

55.2k

82.7k

110.3k

137.9k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

Oracle calls

k

(d) Number of queries for the CIFAR10 dataset.

2

4

6

8

10

27.6k

55.2k

82.7k

110.3k

137.9k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

(f) Number of queries for the Tiny ImageNet dataset.

3.3 Revenue maximization

Consider a company whose objective is to promote a product to users to boost revenue through the 'word-of-mouth' effect. More specifically, given a social network, we need to choose a subset of

up to k users to receive a product for free in exchange for advertising it to their network neighbors, and the goal is to choose users in a manner that maximizes revenue. The problem of optimizing this objective can be formalized as follows. The input is a weighted undirected graph G = ( V, E ) representing a social network, where w ij represents the weight of the edge between vertex i and vertex j (with w ij = 0 if the edge ( i, j ) is absent from the graph). Given a set S ⊆ V of users who have become advocates for the product, the expected revenue generated is proportional to the total influence of S 's users on non-advocate users, formally expressed as f ( x ) = ∑ i ∈ S ∑ j ∈ V \ S w ij . It has been demonstrated that f is non-monotone and submodular [1].

20

40

60

80

100

660

1320

1980

2640

3300

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Function value

(a) Function values for Advogato network dataset.

20

40

60

80

100

133k

265k

398k

530k

663k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

10

15

20

25

30

35

40

45

50

4.59k

6.88k

9.17k

11.47k

13.76k

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Function value

(b) Function values for Facebook network dataset.

10

15

20

25

30

35

40

45

50

0.599M

1.198M

1.796M

2.395M

2.994M

Algorithm 3

Buchbinder et al. 2017

Buchbinder et al. 2014

k

Oracle calls

In our experiment, we compared the performance of Algorithm 3 and the two benchmark algorithms on the Facebook network [42] and the Advogato network [29]. The results of this experiment are depicted in Figure 3. Once again, our algorithm enjoys both better output values and lower standard deviations compared to the benchmark algorithms. Our algorithm uses more queries compared to the Random Sampling algorithm of [6], but the ratio between the number of queries used by the two algorithms tends to decrease as k increases. The behavior of the Random Greedy algorithm of [5] greatly depends on k . For smaller values of k this algorithm requires roughly as many queries as Random Sampling, but for larger value of k it requires significantly more queries than our algorithm.

4 Conclusion

In this work, we have presented a novel algorithm for submodular maximization subject to cardinality constraint that combines a practical query complexity of O ( n + k 2 ) with an approximation guarantee of 0 . 385 , which improves over the 1 / e -approximation of the state-of-the-art practical algorithms. In addition to giving a theoretical analysis of our algorithm, we have demonstrated its empirical superiority (compared to practical state-of-the-art methods) in various machine learning applications. We hope future work will be able to improve the query complexity of our algorithm to be cleanly linear without sacrificing either the approximation guarantee or the practicality of the algorithm.

Acknowledgment

The work of Loay Mualem and Moran Feldman was supported in part by Israel Science Foundation (ISF) grant number 459/20.

References

Eric Balkanski, Adam Breuer, and Yaron Singer. Non-monotone submodular maximization in exponentially fewer iterations. Advances in Neural Information Processing Systems , 31, 2018.

Eric Balkanski, Steven DiSilvio, and Alan Kuhnle. Submodular maximization in exactly n queries. CoRR , abs/2406.00148, 2024.

Niv Buchbinder and Moran Feldman. Constrained submodular maximization via a nonsymmetric technique. Mathematics of Operations Research , 44(3):988-1005, 2019.

Niv Buchbinder and Moran Feldman. Constrained submodular maximization via new bounds for dr-submodular functions. arXiv preprint arXiv:2311.01129 , 2023.

Niv Buchbinder, Moran Feldman, Joseph Naor, and Roy Schwartz. Submodular maximization with cardinality constraints. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms , pages 1433-1452. SIAM, 2014.

Niv Buchbinder, Moran Feldman, and Roy Schwartz. Comparing apples and oranges: Query trade-off in submodular maximization. Mathematics of Operations Research , 42(2):308-329, 2017.

Niv Buchbinder, Moran Feldman, Joseph Seffi, and Roy Schwartz. A tight linear time (1/2)approximation for unconstrained submodular maximization. SIAM Journal on Computing , 44(5):1384-1402, 2015.

Gruia C˘ alinescu, Chandra Chekuri, Martin Pál, and Jan Vondrák. Maximizing a monotone submodular function subject to a matroid constraint. SIAM J. Comput. , 40(6):1740-1766, 2011.

Emmanuel J Candès and Benjamin Recht. Exact matrix completion via convex optimization. Foundations of Computational mathematics , 9(6):717-772, 2009.

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 9650-9660, 2021.

Chandra Chekuri, Jan Vondrák, and Rico Zenklusen. Submodular function maximization via the multilinear relaxation and contention resolution schemes. SIAM J. Comput. , 43(6):1831-1879, 2014.

Yixin Chen, Ankur Nath, Chunli Peng, and Alan Kuhnle. Guided combinatorial algorithms for submodular maximization. arXiv preprint arXiv:2405.05202 , 2024.

Abhimanyu Das and David Kempe. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In ICML , pages 1057-1064, 2011.

Ethan R. Elenberg, Alexandros G. Dimakis, Moran Feldman, and Amin Karbasi. Streaming weak submodularity: interpreting neural networks on the fly. In NeurIPS , pages 4047-4057, 2017.

Alina Ene and Huy L Nguyen. Constrained submodular maximization: Beyond 1/e. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) , pages 248-257. IEEE, 2016.

Uriel Feige, Vahab S. Mirrokni, and Jan Vondrák. Maximizing non-monotone submodular functions. SIAM J. Comput. , 40(4):1133-1153, 2011.

Kai Han, Shuang Cui, Benwei Wu, et al. Deterministic approximation for submodular maximization over a matroid in nearly linear time. Advances in Neural Information Processing Systems , 33:430-441, 2020.

F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (TiiS) , 5(4):1-19, 2015.

Charles R. Harris, K. Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature , 585:357-362, 2020.

Chien-Chung Huang and Naonori Kakimura. Multi-pass streaming algorithms for monotone submodular function maximization. Theory of Computing Systems , 66(1):354-394, 2022.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, Ontario, Canada, 2009.

Alan Kuhnle. Quick streaming algorithms for maximization of monotone submodular functions in linear time. In International Conference on Artificial Intelligence and Statistics , pages 1360-1368. PMLR, 2021.

Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. Numba: A llvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC , pages 1-6, 2015.

Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N , 7(7):3, 2015.

Jon Lee, Vahab S. Mirrokni, Viswanath Nagarajan, and Maxim Sviridenko. Non-monotone submodular maximization under matroid and knapsack constraints. In Michael Mitzenmacher, editor, Proceedings of the 41st Annual ACM Symposium on Theory of Computing (STOC) , pages 323-332. ACM, 2009.

Qi Lei, Lingfei Wu, Pin-Yu Chen, Alex Dimakis, Inderjit S. Dhillon, and Michael J. Witbrock. Discrete adversarial attacks and submodular optimization with applications to text classification. In Ameet Talwalkar, Virginia Smith, and Matei Zaharia, editors, MLSys . mlsys.org, 2019.

Wenxin Li, Moran Feldman, Ehsan Kazemi, and Amin Karbasi. Submodular maximization in clean linear time. Advances in Neural Information Processing Systems , 35:17473-17487, 2022.

László Lovász. Submodular functions and convexity. In A. Bachem, M. Grötschel, and B. Korte, editors, Mathematical Programming: the State of the Art , pages 235-257. Springer, 1983.

Paolo Massa, Martino Salvetti, and Danilo Tomasoni. Bowling alone and trust decline in social network sites. In IEEE International Conference on Dependable, Autonomic and Secure Computing (DASC) , pages 658-663. IEEE Computer Society, 2009.

Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, and Amin Karbasi. Fast constrained submodular maximization: Personalized data summarization. In ICML , pages 1358-1367. PMLR, 2016.

Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and Andreas Krause. Lazier than lazy greedy. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 1812-1818. AAAI Press, 2015.

Marko Mitrovic, Ehsan Kazemi, Morteza Zadimoghaddam, and Amin Karbasi. Data summarization at scale: A two-stage submodular approach. In ICML , pages 3596-3605. PMLR, 2018.

Loay Mualem and Moran Feldman. Using partial monotonicity in submodular maximization. Advances in Neural Information Processing Systems , 35:2723-2736, 2022.

Loay Mualem and Moran Feldman. Resolving the approximability of offline and online nonmonotone dr-submodular maximization over general convex sets. In International Conference on Artificial Intelligence and Statistics , pages 2542-2564. PMLR, 2023.

Loay Mualem, Murad Tukan, and Moran Fledman. Bridging the gap between general and down-closed convex sets in submodular maximization. arXiv preprint arXiv:2401.09251 , 2024.

Loay Raed Mualem, Ethan R Elenberg, Moran Feldman, and Amin Karbasi. Submodular minimax optimization: Finding effective sets. In International Conference on Artificial Intelligence and Statistics , pages 1081-1089. PMLR, 2024.

George L Nemhauser and Laurence A Wolsey. Best algorithms for approximating the maximum of a submodular set function. Mathematics of operations research , 3(3):177-188, 1978.

George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. An analysis of approximations for maximizing submodular set functions-i. Mathematical programming , 14:265-294, 1978.

Ashkan Norouzi-Fard, Jakub Tarnawski, Slobodan Mitrovic, Amir Zandieh, Aidasadat Mousavifar, and Ola Svensson. Beyond 1/2-approximation for submodular maximization on massive data streams. In ICML , pages 3829-3838. PMLR, 2018.

Benjamin Qi. On maximizing sums of non-monotone submodular and linear functions. In Sang Won Bae and Heejin Park, editors, International Symposium on Algorithms and Computation (ISAAC) , volume 248 of LIPIcs , pages 41:1-41:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

Murad Tukan, Fares Fares, Yotam Grufinkle, Ido Talmor, Loay Mualem, Vladimir Braverman, and Dan Feldman. Orbslam3-enhanced autonomous toy drones: Pioneering indoor exploration. arXiv preprint arXiv:2312.13385 , 2023.

Bimal Viswanath, Alan Mislove, Meeyoung Cha, and Krishna P Gummadi. On the evolution of user interaction in facebook. In Proceedings of the 2nd ACM workshop on Online social networks , pages 37-42, 2009.

Jan Vondrák. Symmetry and approximability of submodular maximization problems. SIAM J. Comput. , 42(1):265-304, 2013.

T Wolf. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 , 2019.

Lifeng Zhou and Pratap Tokekar. Risk-aware submodular optimization for multirobot coordination. IEEE Transactions on Robotics , 38(5):3064-3084, 2022.

A A warmup version of our algorithm

In this section, we present and analyze a simpler version of our algorithm with the same general structure, but excluding the speedup techniques used to obtain our main result. Inspired by Buchbinder et al. [3], and similar to Algorithm 3, this simpler version (given as Algorithm 6) comprises three steps: (i) Searching for a good initial solution that guarantees a constant approximation to the optimal set. This is accomplished by running the Twin Greedy algorithm of [17], (ii) Finding an (approximate) local search optimum set Z using a local search method. In this simple version, we use the classical local search algorithm for this purpose, which requires O ( nk 2 ) queries to the objective function. (iii) Lastly, we construct another solution using a version of the Random Greedy algorithm suggested by Buchbinder et al. [5] that avoids elements of the set Z in its first iterations. The algorithm terminates by outputting the better of the two solutions generated in the last two steps.

A.1 Local search

In this section, we present (as Algorithm 4) a simple local search algorithm, which is the algorithm used to implement the first two steps of Algorithm 6. Algorithm 4 begins by finding an initial solution S using the Twin Greedy algorithm of [17]. The algorithm then proceeds as follows: (i) If | S | < k , it checks for an element u such that adding u to S increases the function value by at least (1 + ε 2 k ) f ( S ) . If such an element is found, it is added to S . (ii) If | S | = k , it looks for two elements u ∈ N \ S and v ∈ S such that swapping u and v (i.e., removing v from S and adding u to S ) increases the function value by at least (1 + ε 2 k ) f ( S ) . If such elements exist, the algorithm performs the swap. (iii) If no elements satisfy the previous two conditions, the algorithm checks for an element v such that removing v from S increases the function value by (1 + ε 2 k ) f ( S ) . If such an element is found, it is removed from S . Algorithm 4 continues to search for elements satisfying any of the above three conditions until no such elements exist any longer. When this happens, the algorithm terminates and returns the set S as its output.

The properties of Algorithm 4 are formally established by Theorem A.1.

Theorem A.1. Given a positive integer k , a non-negative submodular function f and an error parameter ε ∈ (0 , 1) , Algorithm 4 returns a set S ⊆ N of size at most k such that

while requiring O ε ( nk 2 ) queries to the objective function.

Proof. Let S denote the output of Algorithm 4, and let E -, E ± , and E + be defined as follows.

E -is the event that there exists v ∈ S such that f ( S -v ) ≥ ( 1 + ε 2 k ) f ( S ) .

E ± is the event that there exists u ∈ N \ S, v ∈ S such that f ( S + u -v ) ≥ ( 1 + ε 2 k ) f ( S ) and | S | = k .

E + is the event that there exists u ∈ N \ S such that f ( S + u ) ≥ ( 1 + ε 2 k ) f ( S ) and | S | < k .

Since the algorithm terminated with the set S , none of the events E + , E -or E ± occurs. To lower bound the value of the set S , we inspect the implications arising from each of these events not occurring.

Implications of E -not occurring. Since E -does not occur, for every v ∈ S , ( 1 + ε 2 k ) f ( S ) > f ( S -v ) . Summing this inequality across every element in v ∈ S \ OPT yields

where the second inequality holds by submodularity of f . By rearrangement, we obtain that

where the last inequality holds since | S \ OPT | ≤ | S | ≤ k .

The above proves the second inequality guaranteed by the theorem. Below we show that the first inequality guaranteed by the theorem is implied either by E + not occurring, or by E ± not occuring, depending on the size of S .

Implications of E + not occurring when | S | < k . Since E + does not occur and | S | < k , for every u ∈ N \ S , it holds that ( 1 + ε 2 k ) f ( S ) > f ( S + u ) . Summing the above inequality across every element in u ∈ OPT \ S yields

where the second inequality follows from the submodularity of f . By rearrangement, we now obtain

where the last inequality holds since | OPT \ S | ≤ | OPT | ≤ k . Averaging Inequalities (3) and (4) gives

which proves the first inequality guaranteed by the theorem in the case of | S | < k .

Implications of E ± not occurring when | S | = k . Since E ± does not occur and | S | = k , for every pair u, v where u ∈ N \ S and v ∈ S , ( 1 + ε 2 k ) f ( S ) ≥ f ( S + u -v ) . Summing the above inequality for every v ∈ S \ OPT and u ∈ OPT \ S , yields that

Observe that

To bound A , note that, by the submodularity of f ,

and hence,

To bound B , we note that the submodularity of f implies that

Combining all of the above yields that

which by rearrangement implies

where the last inequality holds since | OPT \ S | = | S \ OPT | ≤ k . This completes the proof of the first inequality guaranteed by the theorem in the case of | S | = k .

To complete the proof of the theorem, it remains to analyze the query complexity of Algorithm 1. First, we recall that S is initialized on Line 1 of Algorithm 4 by the Twin Greedy algorithm of [17], which gives an approximation ratio c = 1 / 5 using O ( n log k ) queries to the objective function. Each iteration of the loop of Algorithm 4 can be implemented using O ( nk ) queries since there are only O ( nk ) ways to choose u ∈ N \ S and v ∈ S . Let us bound the number L of such iterations. Observe that the function value of the set S in Algorithm 4 increases by a multiplicative factor of 1 + ε 2 k following each iteration of the while loop (except for the last one). Since S is initialized with a solution of value at least cf ( OPT ) , and its value is never larger than f ( OPT ) (because it remains feasible), we get

and rearranging gives us

Combining the above results, we get that the query complexity of Algorithm 4 is upper bounded by O ( n log k ) + O ( nk ) · L = O ( n log k ) + O ( nk ) · O ( k/ε ) = O ( nk 2 ε -1 ) .

A.2 Guided Random Greedy

In this section, we present the Guided Random Greedy algorithm (Algorithm 5), which is the variant of the Random Greedy algorithm of [5] used to implement the last step of Algorithm 6. Algorithm 5 starts with an empty set, and adds to it one element in each iteration until returning the final solution after k iterations. In its first ⌈ k · t s ⌉ iterations, the algorithm ignores the elements of Z , and in the rest of the iterations, it considers all elements. However, except for this difference, the behavior of the algorithm in all iterations is very similar. Specifically, in each iteration i the algorithm does the following two steps. In Step (i) the algorithm finds a subset M i of size k maximizing the sum of marginal gains of the elements u ∈ M i with respect to the current solution S i -1 . In step ( ii ) , the algorithm chooses a random element from M i and adds it to the solution. This algorithm implicitly assumes that |N| ≥ 3 k . If this is not the case, one can fix that by adding to the ground set 2 k dummy elements of value 0 before executing the algorithm (and then removing any dummy elements that appear in the solution of the algorithm).

Algorithm 5: Guided Random Greedy

input : A set Z ⊆ N , a positive integer k ≥ 1 , a non-negative submodular function f , and a flip point t s ∈ [0 , 1]

The properties of Algorithm 5 are given by Theorem A.2.

Theorem A.2. There exists an algorithm that given a positive integer k , a value t s ∈ [0 , 1] , a non-negative submodular function f : 2 N → R ≥ 0 , and a set Z ⊆ N obeying the inequalities given in Theorem A.1, outputs a solution S k , obeying

where α = 1 -1 /k . Furthermore, this algorithm requires only O ( nk ) queries to the objective function.

To prove Theorem A.2, we first need to present some preliminaries. The Lovász extension of f is a function ˆ f : [0 , 1] N → R defined as follows. For every vector x ∈ [0 , 1] N ,

where T λ ( x ) ≜ { u ∈ N | x u ≥ λ } . The Lovász extension of a submodular function is known to be convex. More important for us is the following known lemma regarding this extension. This lemma stems from an equality, proved by Lovász [28], between the Lovász extension of a submodular function and another extension known as the convex closure.

Lemma A.3. Let f : 2 N → R be a submodular function, and let ˆ f be its Lovász extension. For every x ∈ [0 , 1] N and random set D x ⊆ N obeying Pr[ u ∈ D x ] = x u for every u ∈ N (i.e., the marginals of D x agree with x ), ˆ f ( x ) ≤ E [ f ( D x )] .

Using the last lemma, we now prove a lower bound on the expected value of the union of any set A with the solution S i of Algorithm 5 after i iterations.

Lemma A.4. For every integer 0 ≤ i ≤ k and set A ⊆ N , it holds that

Proof. Let x ( i ) ∈ [0 , 1] N be the vector of the marginal probabilities of elements to belong to S i . In other words, for every element u ∈ N , x ( i ) u = Pr[ u ∈ S i ] . Since each iteration of Algorithm 5 adds each element to the solution with probability at most 1 /k , the coordinates of x ( i ) are all upper bounded by 1 -(1 -1 /k ) i . For elements of Z we also know that they are not added by Algorithm 5 to the solution in the first ⌈ k · t s ⌉ iterations, and therefore, their coordinates in x ( i ) are upper bounded by

Let 1 A denote a vector in { 0 , 1 } N containing 1 s at the entries that correspond to elements present in A and 0 in the remaining coordinates. We also denote by x ( i ) ∨ 1 A the coordinate-wise maximum of x ( i ) and 1 A . By Lemma A.3,

where the second inequality holds by the non-negativity of f , and the last inequality follows since T λ ( x S i ∨ 1 A ) = B λ ∪ A for some set B λ ⊂ N \ Z , and the submodularity and non-negativity of f imply together that

With the above result, we are now ready to bound the expected value of f ( S i ) .

Lemma A.5. Let α = 1 -1 . Then, for every integer 0 ≤ i ≤ ⌈ t s · k ⌉

and for every integer ⌈ t s · k ⌉ ≤ i ≤ k

Proof. Let E i be an event fixing all the random decisions in Algorithm 5 up to iteration i -1 (including), and let A i = OPT \ Z for i ≤ ⌈ t s · k ⌉ and A i = OPT for i > ⌈ t s · k ⌉ . Since all the elements of A i can be chosen to be in M i , and so can the dummy elements, we get that, conditioned on E i

where the last inequality holds by the submodularity of f . Since S i = S i -1 + u i , rearranging the last inequality gives E [ f ( S i )] ≥ k -1 f ( S i -1 ∪ A i ) + αf ( S i -1 ) . Taking expectation now over all possible events E i we get that, without conditioning on anything,

Proving the first inequality of the lemma. We prove the first inequality of the lemma (for 0 ≤ i ≤ ⌈ t s · k ⌉ ) by induction on i . For i = 0 the inequality holds by the non-negativity of f . Assume now that 1 ≤ i ≤ ⌈ t s · k ⌉ and the inequality holds for i -1 , and let us prove the inequality for i .

where the second inequality holds by Lemma A.5 and the induction hypothesis since A i = OPT \ Z , and the first equality holds by definition of α .

Proving the second inequality of the lemma. We prove the second inequality of the lemma (for ⌈ t s · k ⌉ ≤ i ≤ k ) by induction on i . One can verify that for i = ⌈ t s · k ⌉ the inequality trivially holds. Assume now that ⌈ t s · k ⌉ < i ≤ k and the inequality holds for i -1 , and let us prove the inequality for i .

where the second inequality holds by Lemma A.5 and the induction hypothesis since A i = OPT , and the first equality holds by definition of α .

We are now ready to prove Theorem A.2.

Proof of Theorem A.2. Note that by submodularity and non-negativity of f ,

The lower bound in the theorem follows by combining the above inequality with the following two inequalities: the inequality arising by plugging i = ⌈ t s · k ⌉ into the first case of Lemma A.5, and the inequality arising by plugging i = k into the second case of Lemma A.5.

To conclude the proof, observe that each one of the k iterations of Algorithm 5 requires us to compute the marginal gain of at most n elements with respect to the set S i -1 , which can be done using O ( n ) queries to the objective function per iteration, and O ( nk ) queries in total.

A.3 0.385-Approximation Guarantee

We are now ready to present Algorithm 6 (the simpler version of our main algorithm). Recall that this algorithm returns the better among the two sets produced in the last two steps described in the beginning of this appendix. Formally, these sets are the output sets of LOCAL-SEARCH and Algorithm 5.

Algorithm 6: Warmup Algorithm

input : A positive integer k ≥ 1 , a non-negative submodular function f , and a flip point 0 ≤ t s ≤ 1 output : A set S ⊆ N

1 S 1 ← LOCAL-SEARCH ( k, f ) .

2 S 2 ← the output of Algorithm 5.

3 return max { f ( Z ) , f ( A ) } .

The following theorem is proved by setting ε in Algorithm 6 to be a small enough positive constant.

Theorem A.6 (Approximation guarantee) . Given an integer k ≥ 1 and a non-negative submodular function f : 2 N → R ≥ 0 , there exists a 0 . 385 -approximation algorithm for the problem of finding a set S ⊆ N of size at most k maximizing f . This algorithm uses O ( nk 2 ) queries to the objective function.

Proof. The proof of the approximation guarantee is very similar to the corresponding part in the proof Theorem 2.3, and is thus, omitted. The query complexity stated in the theorem follows directly from Theorems A.1 and A.2 and the fact that ε is set to a positive constant value.

B Omitted Proofs of Section 2

In this section, we prove the theorems whose proofs have been omitted from Section 2, namely, Theorems 2.1, and 2.2.

B.1 Proof of Theorem 2.1

In this section, we prove Theorem 2.1. We begin with the following lemma. We assume without loss of generality that OPT is of size k (otherwise, we add to OPT dummy elements).

Lemma B.1. If the set S 0 provides c -approximation, then each iteration of the loop starting on Line 3 in Algorithm 1 returns a set with probability at least k/ ( cε (1 -1 / e ) L ) . Moreover, when this happens, the output set S returned obeys

Proof. For every two integers 0 ≤ i < L and 1 ≤ j ≤ ⌈ log 1 ε ⌉ , we denote by A j i the event that the set S j i obeys the condition on Line 13 of Algorithm 1, i.e., the event that for every integer 0 ≤ t ≤ k it holds that

To better understand the implication of an event A j i , assume that such an event occurs, and observe that for t = | OPT \ S j i | , it holds that

where the first equality holds by setting J j i ≜ S j i ∩ OPT , K j i ≜ S j i \ OPT and using u 1 , u 2 , . . . , u | K j i | to denote the elements of K j i in some arbitrary order. The first and last inequalities follow from submodularity of f , the second inequality holds since the fact that S j i and OPT are both of size k implies that t = | OPT \ S j i | = | S j i \ OPT | , and the third inequality holds under the assumption that event A j i occurs. Rearranging the last inequality, we get

In addition, the existence of the dummy elements implies that max S ⊆N\ S j i , | S | = t ∑ u ∈ S f ( u | S j i ) ≥ 0 , and plugging this inequality into Inequality (6) yields that the event A j i also implies

and rearranging this inequality gives f ( S j i ) ≥ f ( S j i ∩ OPT ) 1+ ε .

The above shows that to prove the lemma it suffices to show that the probability that A j i ∗ holds is at least k/ ( cε (1 -1 / e ) L ) . Towards this goal, let us study the implications of the complementary event ¯ A j i . Specifically, we would like to lower bound E [ f ( S j i +1 ) -f ( S j i ) | ¯ A j i ] .

Fix a particular set S j i that causes the event ¯ A j i to occur (notice that the occurrence of this event depends only on the set S j i ). Then, there must exist sets T + ⊆ N \ S j i and T -⊆ S j i of size t ≤ k such that

̸

We can now define the event B j i as the event that Z j i +1 ∩ T + = ∅ (notice that the event B j i is defined only for this particular set S i j ). The probability of the event B j i is

where the first inequality holds since (1 -1 k ) x ≤ e -x k for any x ≥ 0 , and the second inequality holds for any x ∈ [0 , 1] by the concavity of 1 -e -x . By the law of total expectation and the fact that f ( S j i +1 ) is always at least f ( S j i ) , we now get

where the last inequality holds since

where the second inequality holds by submodularity of f ( · ) , the third inequality holds by the way Algorithm 1 chooses u j i and v j i if we let u ′ be a uniformly random element of T + ∩ Z j i +1 and v ′ be a uniformly random element of T -, the penultimate inequality holds by our assumption that S j i implies the event ¯ A j i , and finally, the last inequality holds since it is guaranteed that f ( S j i ) ≥ f ( S j 0 ) = f ( S 0 ) ≥ c · f ( OPT ) .

Since the above bound on the expectation holds conditioned on every set S j i that implies the event ¯ A j i , it holds (by the law of total expectation) also conditioned on the event ¯ A j i itself. Adding this lower bound for all i values, and using the non-negativity of f , we get

Combining the last inequality with the fact that f ( S j L ) is deterministically at most f ( OPT ) , it must hold that

Hence, the probability that the event A j i ∗ does not hold for a uniformly random i ∗ ∈ [ L ] is

Theorem 2.1. There exists an algorithm that given a positive integer k , a value ε ∈ (0 , 1) , and a non-negative submodular function f : 2 N → R ≥ 0 , outputs a set S ⊆ N of size at max k that, with probability at least 1 -ε , obeys

Furthermore, the query complexity of the above algorithm is O ε ( n + k 2 ) .

Proof. As mentioned above, we initialize the set S 0 using the deterministic 1 / 4 -approximation algorithm of Balkanski et al. [2], which uses only O ( n ) queries to the objective function. Thus, c = 1 / 4 in our implementation of Algorithm 1. Let us now set L = ⌈ 2 k cε (1 -1 / e ) ⌉ in Algorithm 1. Then, Lemma B.1 guarantees that every iteration of the outer loop of the algorithm returns a set (obeying the requirement of the theorem) with probability at least 1 / 2 . Hence, by repeating this loop ⌈ log 2 ε -1 ⌉ times, we are guaranteed that Algorithm 1 outputs a set with probability at least 1 -ε . To complete the proof of the theorem, it only remains to bound the number of queries to the objective function that are necessary for implementing it. Each iteration of Algorithm 1 can be

implemented using O ( n/k + k ) queries, and for the above choices of L , Algorithm 1 has only O ε ( k ) iterations. Thus, all the iterations of the algorithm can be implemented using O ε ( n + k 2 ) queries in total. It should also be mentioned that evaluating the condition on Line 13 of the algorithm requires O ( n + k ) queries to the objective, and since this condition is evaluated ⌈ log 2 ε -1 ⌉ = O ε (1) times, all its evaluations require in total only O ε ( n + k ) queries.

B.2 Proof of Theorem 2.2

In this section, we prove Theorem 2.2. We begin by observing that, like in Algorithm 5, each iteration of Algorithm 2 adds each element u ∈ N into the solution with probability at most 1 /k , and furthermore, the first ⌈ t s · k ⌉ iterations of the algorithm do not pick elements of Z at all (see the analysis of Sample Greedy in [6] for a proof of a similar observation that is given in more detail). Given this observation, the proof of Lemma A.4 applies also to Algorithm 2. Thus, this lemma can be used in the proof of the following result.

Lemma B.2. Let α = 1 -1 k . Then, for every integer 0 ≤ i ≤ ⌈ t s · k ⌉ ,

and for every integer ⌈ t s · k ⌉ ≤ i ≤ k ,

Proof. Let E i be an event fixing all the random decisions in Algorithm 2 up to iteration i -1 (including), and let A i = OPT \ Z for i ≤ ⌈ t s · k ⌉ and A i = OPT for i > ⌈ t s · k ⌉ . Since all the elements of A i can be sampled in iteration i , by following the proof of Lemma 13 in the analysis of the Sample Greedy algorithm by [6], one can obtain that, conditioned on E i ,

To be more specific, Lemma 11 of [6] shows that with probability at least 1 -ε the element chosen as u i in iteration i of Algorithm 2 belongs to the k elements with the largest marginal values among the elements that can be sampled in this iteration (if less than k elements can be sampled, dummy elements should added for the purpose of this argument). Let B i denote the set of these k elements. Since the probability of each element of B i to be selected as u i is non-decreasing in f ( u i | S i -1 ) , by Chebyshev's sum inequality, we get

where the second inequality holds since B i contains the k elements with the largest marginals among the elements that can be samples, and A i is a set of up to k such elements; and the last inequality follows from the submodularity of f .

Since S i = S i -1 + u i when f ( u i | S i -1 ) ≥ 0 and S i = S i -1 otherwise, we get f ( S i ) -f ( S i -1 ) = max { 0 , f ( u i | S i -1 ) } . Plugging this observation into the previous inequality, and rearranging gives

where the second inequality uses the non-negativity of f , and the last inequality holds since f ( S i -1 ∪ A i ) ≤ f ( S i -1 ) + f ( A i ) -f ( S i -1 ∩ A i ) ≤ f ( S i -1 ) + f ( A i ) ≤ 2 f ( OPT ) because both S i -1 and

A i are feasible solutions, and thus, cannot have a value larger than f ( OPT ) . Taking expectation now over all possible events E i we get that, without conditioning on anything,

The remaining part of this proof is omitted since it is very similar to the corresponding part in the proof of Lemma A.5, except that the last inequality should be used instead of Inequality (5).

We are now ready to prove Theorem 2.2.

Theorem 2.2. There exists an algorithm that given a positive integer k , a value ε ∈ (0 , 1) , a value t s ∈ [0 , 1] , a non-negative submodular function f : 2 N → R ≥ 0 , and a set Z ⊆ N obeying the inequalities stated in Theorem 2.1, outputs a solution S k , obeying

where α ≜ 1 -1 /k . Moreover, this algorithm requires only O ε ( n ) queries to the objective function.

Proof of Theorem 2.2. The lower bound in Theorem 2.2 follows from the same arguments used in the proof of Theorem A.2, except that Lemma B.2 is used instead of Lemma A.5, which results in the additional error term 2 εf ( OPT ) in the lower bound of Theorem 2.2.

To bound the number of queries to the objective function necessary for implementing Algorithm 2, observe that each iteration of Algorithm 2 samples O ε ( n/k ) elements, and the marginal gain (with respect to S i -1 ) has to be computed only for the sampled elements. Thus, each iteration of Algorithm 2 requires only O ε ( n/k ) queries to the objective function. Since the algorithm has only k iterations, its total query complexity is k · O ε ( n/k ) = O ε ( n ) .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our theoretical contribution can be found in Section 2, while the empirical evaluation of our method can be found in Section 3.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [NA]

Justification: Our paper has no limitations.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proofs of the theoretical results can be found in Section 2, Appendix A, and Appendix B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All the information needed to reproduce the main experimental results of the paper can be found in Section 3.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code can be accessed at the URL https://github.com/muradtuk/ 385ApproximationSubMax .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All the details can be found in Section 3.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We present the error bars as the shaded regions in all of the figures. In Section 3, we clearly explain that the sizes of the error bars are determined by the standard deviations.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [No]

Justification: In Section 3, we wrote the computer type and memory needed to reproduce the experiments. Due to its high dependence on implementation details, we do not provide the time of execution. Instead, we provide in Section 3 the number of queries needed to obtain the results.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our paper studies a well-known submodular optimization problem without societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The creators or original owners of assets (e.g., code, data, models) used in the paper are properly credited in Section 3.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper involves neither crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper involves neither crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of approximately maximizing a submodular function under a constraint that the sets can be of size at most $k$. By carefully combining algorithms from previous work and developing them further, the authors obtain an algorithm that guarantees a $0.385$-approximation with $O_\\epsilon(n + k^2)$ queries to the function and a user-defined probability of failure. The good performance of the proposed algorithm is demostrated with several experiments.\n\nSTRENGTHS:\nEven though the work relies heavily on previous work as described in the beginning of Sec. 3, their combination appears highly nontrivial to me, and the techniques have been developed further from the previous work.\n\nThe empirical results demonstrate clear improvements over previous state of the art and appear to have significantly less variance in the output.\n\nWEAKNESSES:\nBecause of my limited expertise on the topic, my largest complaint is on the presentation of the results: Although the authors do a good job in explaining the ideas behind the math, the formulas are occasionally relatively cumbersome to read, e.g., Line 14 of Algorithm 1. On the other hand, I must admit that I do not have an immediate solution on how to improve them.\n\nQUESTIONS:\nIn terms of the running time, i.e., not the number of queries, how does the proposed algorithm compare against the state of the art? Because the running time depends on, for example, $epsilon$, it is not immediately clear to me how large the constant factors of the algorithms would be.\n\nOther:\n- Abstract: Without previous background on the topic, it was unclear to what $k$ and $n$ referred to in the abstract.\n- Lines 29–32: first, $B$ is a subset of $A$, but then $A$ is a subset of $B$. This can be confusing to the reader.\n- Line 66: If I understood the paper correctly, I think it would be more precise to use $O_\\epsilon$ instead of the $O$-notation here (although a constant $\\epsilon$ is used in the experiments)\n- Caption of Alg. 3: maximiziation -> maximization\n- Line 172: OPT is not written with \\mathbb\n\nLIMITATIONS:\nSince the paper focuses on a optimizing a method that is used in ML applications and does not study application directly, I think the limited discussion of limitations is sufficient here.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-25"}

comments: SUMMARY:
The paper introduces a new algorithm, FAST-LOCAL-SEARCH, for maximizing non-monotone submodular functions, achieving a 0.385-approximation with low query complexity. It combines initial solution search, accelerated local search, and stochastic greedy improvement steps to outperform existing algorithms in machine-learning applications like movie recommendation, image summarization, and revenue maximization. The algorithm guarantees a constant approximation to the optimal set and is supported by theoretical proofs and empirical evaluations.

STRENGTHS:
The paper introduces an algorithm, FAST-LOCAL-SEARCH, for maximizing non-monotone submodular functions under a cardinality constraint. This algorithm combines initial solution search, accelerated local search, and stochastic greedy improvement steps to achieve a 0.385-approximation with query complexity of O(n + k^2). The practical performance of the algorithm has been validated in real-world applications.

WEAKNESSES:
The novelty of the paper is limited. The idea of this paper is derived from existing work [2], and more significantly, the "guided algorithm" proposed in this paper is very similar to [10] in both algorithm design and theoretical analysis.

The authors claim that the algorithm proposed in this paper has a practical query complexity of O(n+k^2); however, in practical applications, the value of k is often very large, making an O(k^2) query complexity potentially impractical.

QUESTIONS:
Please refer to weaknesses.

LIMITATIONS:
N.A.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work addresses the problem of maximizing a non-monotone submodular function subject to a cardinality constraint. In submodular maximization, we have a set of elements (ground set) and a function that assigns a value to any subset of elements. A function is submodular if adding an element to a smaller set contributes more value than adding it to a larger set. Formally, for sets A and B where A is a subset of B, adding an element x to A increases the function value more than adding x to B. A function is monotone if its value always increases with set size. Non-monotone functions don't necessarily have this property. The goal here is to find a subset of size at most k from the ground set that maximizes the value of the non-monotone submodular function.\n\nWhile they developed a 0.385-approximate solution for this problem, there is an algorithm with a better approximation factor of 0.401. Another work also achieves a 0.385-approximate solution, but both are impractical due to the high number of query calls required. In contrast, this work uses only O(n+k^2) query calls. Additionally, there are two works with a 0.367-approximation, one requiring O(nk) query calls and the other O(n) query calls. In the experiment section, the authors compared the output value of their algorithm with those of the two latter algorithms.\n\nThe core idea of the proposed algorithm involves:\n- Running the sample greedy algorithm by Buchbinder et al. [5].\n- Applying a local search technique to improve the initial solution.\n- Employing a Stochastic greedy method and returning the solution with the highest value between the local search and Stochastic greedy outputs.\n\nThis approach leverages existing techniques while incorporating additional steps to enhance the solution within a reasonable computational cost.\n\nSTRENGTHS:\nFor non-monotone submodular maximization under a cardinality constraint, their work achieves the best query complexity for algorithms that attain at least a 0.385-approximate solution. In other words, among practical algorithms—those that can be implemented and executed in a reasonable time—they offer the highest approximation factor.\n\nWEAKNESSES:\nSince their algorithm does not have the best approximation factor, it is less interesting for theoretical work and more important for practical applications. Therefore, a comprehensive experiment supporting their claims is crucial. While they did a great job by running experiments on different problems and demonstrating the advantage of their output value compared to two other algorithms with worse approximation guarantees, it is also necessary to plot the number of query calls for these algorithms. Comparing the number of query calls in addition to the output value is essential, as they claimed to have the best approximation algorithm with \"a low and practical\" query complexity.\n\n### Comments for the authors:\n- Line 43: Remove “over”\n- Line 50: I think [14] present a ½-approximation algorithm for symmetric submodular functions and for non-symmetric, they only achieve a ⅖-approximation algorithm.\n- Line 68: Do you mean “3 applications”?\n- Line 149: Remove “)” after “pn”\n- Line 208: Replace “she is” with “they are”\n- Line 548: Replace “lemmata” with “lemmas”\nThe revenue maximization experiment is mostly known as the max-cut problem. I think it is good to at least mention that.\n- In Algorithm 3, the term “AIDED-MEASURED DISCRETE STOCHASTIC GREEDY” has been used to refer to Algorithm 2. However, the name of Algorithm 2 is different. \n- From lines 95 and 96, I assumed that Algorithm 2 improves the output of Algorithm 1. However, after reading the algorithms, it seems that Algorithm 2 finds a different\n\nQUESTIONS:\n- What is the query complexity of your algorithm in terms of epsilon? \n- Do you have any practical comparisons of the number of query calls used in your experiments?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a practical $0.385$-approximation algorithm using $O(n+k^2)$ queries for non-monotone submodular maximization under a cardinality (size) constraint, where $n$ is the number of elements in the ground set and $k$ is the maximum size of a feasible solution. As a comparison, the state-of-the-art algorithms attain a $0.401$ approximation ratio with a high number of queries, or $1/e-\\epsilon$ approximation ratio using $O_{\\epsilon}(n)$ queries. The paper also evaluates its method experimentally on several applications.\n\nSTRENGTHS:\n1.\tThe paper makes a substantial progress for the studied problem by presenting a new algorithm with low query complexity and improved approximation ratio. The algorithm is obtained by carefully combining several well-known techniques and methods in the field. Such a result already contains enough originality as a NeurIPS submission.\n2.\tThe paper is complete, in the sense that it contains both a correct theoretical analysis and detailed experimental evaluations of its algorithm.\n3.\tThe paper is organized well. It splits the algorithm into several ingredients and then explains each part in detail and how to combine them to get the final result. In such a way, it’s easy for readers to follow the basic ideas and check the correctness of the algorithm.\n4.\tThe paper, as suggested by the experiments, provides a better solution for several important applications. On the other hand, it further exploits several mature techniques in the field, which may inspire more results in the area.\n\nWEAKNESSES:\n1.\tThere are many typos in the proofs (see Questions for some examples), which affects the clarity. The authors are suggested to proofread their paper in a later version.\n2.\tThe main algorithmic framework is borrowed from [2]. So, if the space is enough, I suggest the authors to add a paragraph discussing why the framework can beat the $1/e$ ratio. In doing so, the paper might be more readable to those who haven’t read [2] before.\n\nQUESTIONS:\n1.\tI’m curious about if it’s possible to discretize the algorithm in [3] to get a practical $0.401$ algorithm, just like this paper.\n2.\tFollowing are some typos:\n1) I suggest using “at most” to replace “at max” in the paper.\n2) Page 13. “Let S be a subset in N of size at (missing most) k”.\n3) Page 16, Lemma A.3, $x_{\\in}$ --> $x\\in$.\n4) Page 16, in the last inequality, \\beta^{i} should be \\beta_{i}.\n5) Page 17, in the first formula, A\\cup B_{\\lambda}\\cap Z should be A\\cup B_{\\lambda} ‘\\cup’ Z.\n6) Page 18, Lemma B.1, “that starts returns a set…” doesn’t read smoothly.\n7) Page 18, in the first sentence of the proof of Lemma B.1, S_{i,j} should be S_i^j.\n8) Page 20, in inequality (6), the subscript L should be i.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-13"}