Practical $0.385$-Approximation for Submodular Maximization Subject to a Cardinality Constraint
Authors: Murad Tukan
Loay Mualem
Moran Feldman
Venue: NeurIPS 2024 poster (2024)
Keywords: Submodular maximization
Discrete optimization
Machine learning
Primary area: optimization
TL;DR: In this work, we present a novel algorithm for submodular maximization subject to a cardinality constraint that combines a guarantee of $0.385$-approximation with a low and practical query complexity of $O(n+k^2)$.

Abstract: Non-monotone constrained submodular maximization plays a crucial role in various machine learning applications. However, existing algorithms often struggle with a trade-off between approximation guarantees and practical efficiency. The current state-of-the-art is a recent $0.401$-approximation algorithm, but its computational complexity makes it highly impractical. The best practical algorithms for the problem only guarantee $1/e$-approximation. In this work, we present a novel algorithm for submodular maximization subject to a cardinality constraint that combines a guarantee of $0.385$-approximation with a low and practical query complexity of $O(n+k^2)$. Furthermore, we evaluate our algorithm's performance through extensive machine learning applications, including Movie Recommendation, Image Summarization, and more. These evaluations demonstrate the efficacy of our approach.

Practical 0.8o--Appooximmtion for Submodular Maximization Subject to a Cardinality Constraint
Murad Tukan DataHeroes Israel muradedataheross.c
Loay Mualem
Department of Computer Science University of Haifa
Haifa Israel
1oaymual@gmail........aa com
Moran Feldman
Department of Computer Science University of Haifa
Haifa Israel moranfecce hwifa.aiaa..........a ac. il
Abstract
Non-monotone constrained submodular maximization plays crucial role in various machine learning applications. However, existing algorithms often struggle with a trade-off' between approximation guarantees and practical efficiency. The current state-of-the offotff-eartttt a recent 401-approximation algorithm, but its computational complexity makes it highly impractical. The best practical algorithms for the problem only guarantee 1/ j/e-approximation. In this work, we present novel algorithm for submodular maximization subject to a cardinality constraint that combines guarantee of 1.387-approximation with a low and practical query complexity of (n=+k2), where is the size of the ground set and k is the maximum size of a feasible solution. Furthermore, we evaluate the empirical performance of our algorithm in experiments based on the machine learning applications of Movie Recommendation, Image Summarization, and Revenue Maximization. These experiments demonstrate the efficacy of our approach.
Introduction
In the last few years, the ability to effectively summarize data has gained importance due to the advent of massive datasets many fields. Such summarization often consists of selecting small representative subset from a large corpus of images, text, movies, etc. Without specific structure, this task can be challenging as finding global minimum of a non-convex function. Fortunately, many practical machine learning problems exhibit some structure, making them suitable for optimization techniques (either exact or approximate).
key structure present many such problems is submodularity, also known as the principle of diminishing returns. This principle suggests that the incremental value of element decreases as the set it is added to grows. Submodularity enables the creation of algorithms that can provide near-optimal solutions, making it fundamental in machine learning. It has been successfully applied various tasks, such as social graph analysis [39], adversarial attacks 26 36), dictionary learning [13].
data summarization [32 34 35). interpreting neural networks [14) robotics 45 41 and many more.
To exemplify the notion of submodularity, consider the following task. Given a large dataset, our goal is to identify subset that effectively summarizes (or covers) the data, with good representative
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

set being one that covers the majority of the data. Note that adding an element to a set B is less beneficial to this goal than adding it to a subset A C due the higher likelihood of overlapping coverage. Formally, if N is the set of elements in the dataset, and we define function f: 2 + IR mapping every set of elements to its coverage, then, the above discussion implies that, for every two sets A S B C N and element E N B, it must hold that f(s A) 2 f(s | B), where [(s A) _ [([)) - f(A) denotes the marginal gain of the element with respect to the set A. We say that a set function is submodular if it obeys this property.
Unfortunately, maximizing submodular functions is NP-hard even without a constraint [6], and therefore, works on maximization of such functions aim for approximations. Many of these works make the extra assumption that the submodular function f: 2 - R is monotone, i.e., that for every two sets A C B C .N, it holds that f(B) f(A). Two of the first works of this kind, by Nemhauser and Wolsey [37] and Nemhauser et al. [38.. showed that greedy algorithm achieves a tight 1 - 1/c approximation for the problem maximizing a non-negative monotone submodular function subject to cardinality constraint using ((n)) function evaluations, where is the size of the ground set ' and k is the maximum cardinality allowed for the output set. An important line of work aimed to improve the time complexity of the last algorithm, culminating with deterministic and randomized algorithms that have managed to reduce the time complexity to linear at the cost of an approximation guarantee that is worse only by a factor of 1 66 31 27 20].
Unfortunately, the submodular functions that arise in machine learning applications are often non- monotone, either because they are naturally non-monotone, or because monotone regularizer is added to them. Maximizing a non-monotone submodular function chal- lenging. The only tight approximation known for such functions is for the case of unconstrained maximization, which enjoys a tight approximation ratio of 1/2 [16 77. slightly more involved case is the problem of maximizing a non-negative (not necessarily monotone) submodular function subject to cardinality constraint. This problem has been studied extensively. First, Lee et al. (25) suggested approximation for it. This approximation ratio was improved in leading to a very recent 0,401 0,401-approxim -pprrxxxxxnnnno algorithm due to Buchbinder and Feldman 4 which improved over a previous 1.385-approximation algorithm due to
promoting non-
an algorithm guarantecing (1/4
a long series works [
15
43
Buchbinder and Feldman 3 On the inapproximability side, it has been shown that no algorithm can
guarantee better approximation ratio than 0.478 in polynomial time (40)
Most of the results in the above-mentioned line of work are only of theoretical interest due to a very high time complexity. The two exceptions are the Random Greedy algorithm of Buchbinder et al. [5] that guarantees \/-approxxmmtion using (nkk) queries to the objective function, and the Sample Greedy algorithm of Buchbinder et al. [6] that reduces the query complexity to Og (n) at the cost of slightly worse approximation ratio of a/e- E.
1.1 Our contribution
In this work, we introduce novel combinatorial algorithm for maximizing non-negative submodular function subject to a cardinality constraint. Our suggested method combines a practical query complexity of On + k²) with an approximation guarantee of 0.385, which improves over the 1/-aaprrxximatio of the statc-of-the-art practical algorithm. To emphasize the effectiveness of our suggested method, we empirically evaluate it on applications: (i) Movic Recommendation. (ii) Image Summarization, and (iii) Revenue Maximization. Our experiments on these applications demonstrate that our algorithm (Algorithm 3) outperforms the current practical offte-of-the-ar algorithms.
Remark. An independent work that recently appeared on arXiv [1] suggests another 0.385- approximation algorithm for our problem using O(nk) oracle queries. Interestingly. their algorithm very similar to basic version of our algorithm presented in Appendix A In this work, our main goal is to find ways to speed up this basic algorithm. which leads to our main result. In contrast, the main goal of [12 is to derandomize the basic algorithm and extend other constraints.
1.2 Additional notation
Let us define some additional notation used throughout the paper. Given an element E / and set S C A, we use + u and as shorthands for S SU (u) and S {u), respectively. Given also a set function f: 2 + H. we recall that f(u (S) is used to denote the marginal contribution of u to

---

S. Similarly, given an additional set T N, we define f(T 5) _ f(SUT) f(S). Finally, we denote by CPT an arbitrary optimal solution for the problem we consider.
2 Method
In this section, we present our algorithm for non-mononn monotone submodular maximization under car- dinality constraints, which the algorithm used to prove the main theoretical result of our work (Theorem 2.3). We begin with a brief overview of our algorithm. Motivated by the ideas underlying the impractical 0-385-approximmation algorithm of our algorithm comprises three steps:
1. Initial Solution: We start by searching for good initial solution that guarantees constant approximation to the optimal set. This is accomplished by running the recent deterministic 1/ /-approximmatin algorithm of Balkanski et al. 121
2. Accelerated Local Search (Algorithm D]: Next, the algorithm aims to find an (approximate) local optimum set using a local search method. This can be done using a classical local search algorithm the cost of O₂ (nk²) queries (see Appendix A for more detail). As an alternative, we introduce, in Subsection 2.T our accelerated Tocal search algorithm FAST-LOCAL-SEARCH (Algorithm D which reduces the query complexity to Oz + 1²).
3. Accelerated Stochastic Greedy Improvement (Algorithm) 2): It can be shown that when the set Z does not have good value, it contains only little of the value of the optimal solution, and at the same time, it contains many of the elements that negatively affects this optimal solution. Thus, it makes sense try to avoid this set. Accordingly, after obtaining the set Z, our algorithm constructs a second possible solution using a stochastic greedy algorithm that picks only elements of / in its first iterations. One can use for this purpose a version of the Random Greedy algorithm suggested by Buchbinder et al. [5] that uses O(nk) queries (see Appendix A for details). To get the same result using fewer queries, we employ Algorithm 2 (described in Subsection 2.2) which is accelerated using ideas borrowed from the Sample Greedy algorithm of 6
Our final algorithm (given Algorithm Subsections 2.3 returns the better among the two sets produced in the last two steps (i.e., the output sets of Algorithm I and Algorithm ()) Intuitively, this algorithm guarantees our target approximation ratio of 0.385 because when f(Z) smaller than this value, the set Z is bad enough that avoiding it (in the first iterations) allows Algorithm to get good enough solution.
2.1 Fast local search
In this section, we present our accelerated local search algorithm, which is the algorithm used to implement the first two steps of our main algorithm. The properties of this algorithm are formally given by Theorem 2.1 Let CPT be an optimal solution.
Theorem 2.1. There exists an algorithm that given a positive integer k, value e E (0, 1), and non-negative submodular function f: 2 + R2o, outputs set S S N of size at max k that, with probability at least E, obeys
f(S) Z CPS) + +(SUOPT) and f(S) f(SNOFT)
2+6
1+s
Furthermore, the query complexity of the above algorithm is On ( k2))
Note that the guarantee of Theorem 2.1 is similar to the guarantee of a classical local search algorithm (see Appendix for details). However, such classical local search algorithm uses O₂ (=(k2) queries, which is higher than the number of queries required for the algorithm from Theorem 2.1
We defer the formal proof of Theorem 2.1 to Appendix B However, we note that this proof is based on Algorithm I Algorithm implicitly assumes that the ground set A includes least k ++ dummy
'A previous version of this paper used for this purpose the randomized Sample Greedy algorithm of [ Since this algorithm is randomized, to get a good solution with a high enough probability, that previous version had to run this algorithm Ollog = times and select the solution with the highest function value. The code in the supplemental material of this paper includes the option use either of these initialization options.
3

---

elements that always have zero marginal contribution to f. Such elements can always be added to the ground set (before executing the algorithm) without affecting the properties of and removing them from the output set of the algorithm does not affect the guarantee of T'eorem 2.1
Algorithm 1: FATT-LOCAA--FFARCCCC.....
input :A positive integer A: 1, submodular function f', an approximation factor E (0, 1), and number I. of iterations.
output: :A subset of A of cardinality at most
Initialize So to be feasible solution that with probability at least - ε provides c-approximation for the problem for some constant с (0, 1].
Fill So with dummy elements to ensure |Sol = k.
for j 1 to flog2 1 do
Let S/6 Sg.
for i to do
7 + Sample x items from N uniformly at random.
ud < mrgmax fu, 1
if fu, | S 0 then ui +- dummy element that does not belong to S
of + arg argmin we f(v') 1 -').
(() s() of u²) then 8 S of ur.
10
else S + S
11
12
Pick uniformly random integer 0 < 8" < L. if for every integer O S 1 k holds that Eves f(u 5%-)
max SEN S.N.5/-.15. 1S| =
Evs f(v 1 - v) +ff(S)) then return S!.. SC8/-
min
14 return FAILURE.
Algorithm| starts by finding an initial solution So guaranteeing constant approximation (we imple- ment this step using the deterministic 1/--aprroximation algorithm of Balkanski et al. (2)) If the size of the initial solution is less than (i.e., |Sal < k), the algorithm adds to it k - |So dummy elements. Then, Algorithm makes roughly log2 e attempts find good output. Each attempt trys to improve the (same) initial solution using L iterations. Each iteration consisting of three steps: In Step (i), the algorithm samples items, and picks the element from the sample with the largest marginal contribution to the current solution S,-1- If there are no elements in the sample with positive marginal contribution, the algorithm picks a dummy element outside S as u. Step (ii), the algorithm picks the element E 5,-1 that has the lowest marginal value, i.c., the element whose removal from S,-1 would lead to the smallest drop in value. In Step (iii), the algorithm swaps the elements u and v if such swap increases the value of the current solution. Once T. iterations are over, the algorithm picks a uniformly random solution among all the solutions seen during this attempt (recall that the algorithm makes roughly log2 e attempts to find good solution). If the random solution found obeys the technical condition given on Line 13 then the algorithm returns it. Otherwise, the algorithm continues to the next attempt. If none of the attempts returns set, the algorithm admits failure.
2.2 Guided stochastic greedy
In this section, we prove Theorem 2.2 which provides the last step of our main algorithm.
Theorem 2.2. There exists an algorithm that given a positive integer k, a value E (0, 1). a value t. [0, 1], a non-negative submodular function f: 2 + R20, and set Z S N obeying the inequalities stated in Theorem 2.1 outputs a solution Sk, obeying
[S{)) 2
k
[t. k.
k- - [6, a - I(OPT)+
k
a + k-
2k
[t. k. k [to -
1(OPTU
+
(a &
a² ) Z) 23 20F(T)T)

---

where a - 1 1/k. Moreover, this algorithm requires only (n(n) queries to the objective function.
The algorithm used to prove Theorem 2.2 Algorithm 2 This algorithm starts with an empty set and adds elements to it in iterations (at most one element per iteration) until final solution is ready after k iterations. In its first [, iterations, the algorithm ignores the elements of Z, and in the other elements. However, except for this difference, the behavior the algorithm in all iterations is very similar. Specifically, in each iteration the algorithm does the following two steps. In Step (i), the algorithm samples a subset M, containing Og (n/k) elements from the data. In Step (ii). the algorithm considers subset of M, (of size either $1 [p(n [21)] or $2 [pn]) containing the elements of Mj with the largest marginal contributions with respect to the current solution S,-1. and adds uniformly random element out of this subset to the solution (if this element has positive marginal contribution).
iterations, considers
Algorithm 2: Guided Stochastic Greedy
input :A set Z C N. a positive integer k 2 1, values (0, 1) and t, [0, 1], and non-negative submodular function f
output:A set Sp S N
Initialize So 5- 01
Define < min (, 8k - (nnss (1) and 82 f- k/n.
3 Define 81 - k/(n
for i = to [k 1, do
Let M, N Z be a uniformly random set containing [p- (n [ clements. Let d, be uniformly random scalar from the range
(n [.
Let u, be an element of M, associated with the dd 1-th largest marginal contribution to S (if
[d,] [,M.). we set u; to be a dummy element having marginal contribution to f).
8
if f(u 1 2 then S, + S {u:}.
10
else
S; +- -S
for i i= [k t. + to k do
13
Let M, N be uniformly random set containing [p [p-n] elements.
Let d; be uniformly random scalar from the range (0, 82 [n]].
15
Let u, be an element of M, associated with the [d, 1tt largest marginal contribution to S;-1-
if f(u (SS-1) then S; S U (u,). else S, < St-1-
17
18
return St-
2.3 0.385-Apprxximation guarantee
In this section, our objective is to prove the following theorem.
Theorem 2.3. Given an integer l: 2 and a non-negative submodular function f: 2 - Rzo. there exists 0.385-approximation algorithm for the problem of finding S N of size most maximizing f. This algorithm uses ((n + k.2) queries to the functiionn
function.
The algorithm used to prove Theorem 2.3 is Algorithm 3 Our technical guarantee for Algorithm is given Lemma 2.4 When & is large enough, this lemma immediately implies Theorem 2.3 by choosing to be a small enough positive constant. If k is small, getting Theorem 2.3 from Lemma Lemma requires three steps process. First, we choose an integer constant p such that pk large enough, and we create a new ground set N,, = {u: E i E [p]] and a new objective function g: 2N,, + R defined as gS)) ([(()))))), where R(S) is a random subset of A that includes every element Š that provides a..8-appproximation for the problem max (AS) |S| S pk)). Finally, the Pipage Rounding technique of S can be used to get from Š 0.385-approximmttin for our original problem. Notice that since the size of Š is constant (as we consider the case of a small k), this rounding can be done using a constant number of queries the objective.
N with probability ( x [p))//p. Then, we use Lemma 2.4 get a set
Lemma 2.4. Algorithm 3 makes O~(n + k.2) queries to the objective function, and returns a set whose expected value is at least (c O(E + k- (-))) for some constant >> 0.385.
5

---

Algorithm 3: A 0.385-approximmaion algorithm for submodular maximization
input :A positive integer k 2 1, a non-negative submodular function f. error parameter e E (0. 1),
and flip point 0 I i, S
output:A set St C
f, =- -1a6(11110) (2k/(((111))) 2 the last algorithm did not fail then
Z < FAST-LOCAL-SEARCIIE
A GUIDED- POOCIAASTICGRREE"Y (Z, k, t),
return the set maximizing f among Z and A.
5 else return 0.
Proof. According to the proof of Theorem 2.1 our choice of the parameter L in Algorithm | guarantees that with probability at least 1 the set Z obeys the inequalities
f(Z) 2 fiZu (PT
n(P))
f(ZNOPT)
and
f(Z) 2
2+ &
1 1+8
Let us denote by & the event that these inequalities hold. By Theorem 2.2
[4. k]
B[f(A) 2
k -- + a k- (1 a² ) S(OPT)+
+ (at + ta 22 [1. .k] a² (e-- E[S(OPT U Z)
| E]
k
+ (a - -kk [E (E n Z) E] (0
Since the output of Algorithm i the better set among A and Z, we can lower bound its value by any convex combination of lower bounds on the values of A and Z. More formally, if we denote by Pl. P2 and 23 any three non-negative values that add up 1, then we get
E mmax (S(A),
2 Pa
lta ki
a
k-
+ k- Tex ) ((OPT)
k
2k - -It a (t- (BT))BB(CC [L U 2) |8]
+
P1
Pa (a k + a
k-1
(1)
2+e E
k
P2 P1 ps (a*
( Z) | [ 25pa. (OPT)
11 2+
To simplify the above inequality, we need to bound some of the terms in it. First,
k It.
k] a k- 11 + a² kk
a >
1 To
((1-)))) a
1 1)
1-t,
2 - t, -
c -1
k -e-
k
3
2 - t,
e d 1 c 2 22 - t,
(2-t, - -
where the first inequality holds since [t. -k] S t.. k ++ 1. a I 1, the second inequality follows since
((1-t,)) 2 c11-1(11
- 1-t, and the last inequality holds since e < 1. Second,
2k- [t. kl k- 1 x1 > 2e
2k - La k
k k 2e-1
k + a k-1
k a
k
k
8 + 21
-1 (2 1a)e
8 + 2e
- 11 -1 (2 - 11g - -2c -1))
k.
where the first incquality holds since = 2 a 2 e (1 (1-1) t and the second inequality holds
since a 1-2 < e.--1 +4/k. Finally, it holds that a a k- [t > -1(11 - /(1 (1-11 2
e +) - + 2 -et--l(1- (1- -
2

---

Plugging all the above lower bounds into Inequality 1 yields the promised simplified guarantee that
(())
8
2 P3
(2 - is -e- (e:))ee-1((000 ((0T)
((OPT) - 0( + k (OOFT)
+
P1
-aee--1(2 -2) - -O(K
2+
P2
P1
D36 -1 (1 --)) - -(k (((P)))E((PPTTRR) - |8|
+
2+
By [3). for an appropriate choice of values for p1 P2, Pa and Eg the last inequality implies
8] 2 (c O( + k + (OPTT)
-Okk-) (OSTP Z) + H(OPT n 2)) (2)
for some constant 0.385. To get from the last inequality the bound on (f f(Z) E] stated in the lemma, we need to show that the conditioning on & and the last term of the inequality can both be dropped. To see why the conditioning can be dropped, note that the event & happens with probability at least 1- - E, and when it does not happen the set returned by the algorithm still has non- negative value. These observations show together that removing the conditioning on & in Inequality (2) only affects the constant inside the big O notation. Notice now that since OPT n Z S CPT is always a feasible solution, deterministically holds that (OPT Z) < (OPP)) Similarly, since
Z is a feasible solution, the submodularity of f guarantees that F(CPT U Z) + f(PP 0 2) <
(CPT) f(Z) S (OPT).. These two bounds allow us to drop the last term of Inequality the cost of increasing (again) the constant inside the big O notation.
at
To complete the proof of the lemma, note that Line of Algorithm 3 requires O2 (n + k2) queries to the objective function as shown in the proof of Theorem 221 while Line 3 of Algorithm requires
Og (n) queries to the objective function as dictated by Theorem 2.2
3 Experiments
To emphasize the effectiveness of our suggested method from Section in this section, we empirically compare Algorithm 3 with two benchmark algorithms on three machine-learnnng applications: movie recommendation, image summarization, and revenue maximization. Each one of these applications necessitates maximization a non-monotone submodular function. The benchmark algorithms we consider are the Random Greedy algorithm of Buchbinder et al. [5], and the Random Sampling algorithm of [6). These algorithms are the current state-of-the-an practical algorithms for maximizing non-monotone submodular functions.
As stated, 2 requires O ((2ine" queries to the objective function, where the dependence on e comes from the choice of value for the parameter p of the algorithm. However, we have found out that in practice a more modest choice of value for p suffices. Specifically, our experiments, we have replaced Line 2 of Algorithm 2 with p min(1, (1, kez ). Throughout the experiments, we have set e = 0.1; and all the reported results are averaged across executions. We use shades in our plots to depict the standard deviations of the individual results obtained in these 8 executions.
&o
Software/Hardware. Our algorithms were implemented in Python 3.11 using mainly "Numpy" 19 and Numba [23. The implementations' code can be found at lattps:/// //11 thub. com/ muradtuk $854pproxiiatiionuublee. The experiments were performed on 2.2GHz 13-338 (24 cores total) machine with 64GB RAM.
3.1 Personalized movie recommendation
Consider movie recommendation system which each user specifies what genres they are interested in, and the system has to provide representative subset of movies from these genres. Assume that each movie is represented by vector consisting of users' ratings for the corresponding movie. One challenge here is that each user does not necessarily rate all the movies. Hence, the vectors representing the movies do not necessarily have similar sizes. To overcome this challenge, low-rank matrix completion techniques [9] can be performed on the matrix with missing values to obtain complete rating matrix. Formally, given a few ratings from users to n movies we obtain this way rating matrix M of size k n. Following 33 30), to score the quality of selected subset of
7

---

movies, we use the function f(S) EueN EvEs Su, -u of n movies, À E [0, 1] is parameter and fa,e denotes the similarity between movies u and (the similarity Su,v can be calculated based the matrix M in multiple ways: cosine similarity, inner product, etc). Note that the first term in f's definition captures the coverage, while the second term captures diversity. Thus, the parameter controls the importance diversity in the returned subset. For any À v 0.5, (S) is monotone
Eves Sa.v Here, . is the set
however, it can be non-monotone for larger values of X.
We followed the experimental setup of the prior works 33 30 and used a subset of movies from the MovieLens data set [18] which includes 10,437 movies. Each movie in this data set is represented by ratings, and used the inner product similarity
25 dimensional feature vector calculated using to obtain the similarity values 8a.e based on these vectors. When experimenting with this application, fixed À to be either 0.55 or 0.75, and varied k:
12. 42k
Function value
9. 8.8kk
M
30.76
V
7.85k
Function tintion
1.1
Algorithe euchbindor
5.35k
20
60
80
100
26
40
60
80
1000
(a) Function values for 0.75.
(b) Function values for À = 0.55.
1.862N
* -AAlgur stite 3
1.86.4
Alguur LElE Buchbinder et al. 2017 -Ruchb-nder et a1. 2014
-BBuchbinder at al. 2017 - -Ruchb-nnder a1. 2014
Gracle calls
9,85M
Oracle callss
9.85M
0.630N
0.630N
0.425M
6.425M
0.233M
0.213N
29
4.
69
89
1.00
28
4.
69
58
198
k
(c) Number of queries for À = 0.75.
(d) Number queries for 0.55.
Figure 1: Experimental results for Personalized Movie Recommendation. Plots and (b) compare the output of our algorithm with the benchmark algorithms mentioned at the beginning of Section for partieular value of the parameter À and varying number of movies. Plots and compare the number of queries used by the various algorithms.
The results of these experiments depicted in Figure 1 One observe that our proposed method, Algorithm demonstrates superior performance compared to the other methods. Moreover, this performance is stable, and presents much smaller variance compared to the variance in the performance of the two benchmark algorithms. The number of queries used by our algorithm is only slightly larger than the number of queries used by the Random Sampling algorithm of [6], and is typically smaller than the number of queries used by the Random Greedy algorithm of Buchbinder et al. [5], sometimes by as much as a factor of
3.2 Personalized image summarization
Consider setting which we get as input a collection A' of images from & disjoint categories (e.g.. birds, dogs, cats) and the user specifies E [f] categories, and then demands subset of the images in these categories that summarizes the images the categories. Following (30) again, to evaluate given subset of images,
the function f(S) EmEN max,es Su, [NT Enes Eves Su,v
where 8a. is non-negative similarity between images and
To obtain the similarity between pair of images u, u, we utilized the VITBI model 10) from HuggingFace [44] as the feature encoder for vision datasets. Specifically, the final layer CLS token

---

embedding output was used as the feature representation. The similarity between pairs of images was then computed as the cosine similarity of the corresponding embedding vectors. To experiment in this setting, we used three datasets: (i) CIFARIO (21 A dataset of 50,000 images belonging to 10 different classes (categories). (ii) CIFARTOO 21
dataset of 50,000 images belonging 100 different classes. (iii) Tiny ImageNer [24] - A dataset of 100,000 images belonging to 200 different classes. each one of our experiments, the task was to summarize set of 10,000 images sampled uniformly from one of these datasets. The upper bound k the number images allowed in the summary varied between experiments. The results our experiments are depicted in Figure 2
Similarly to Section 3.1 we observe that our algorithm (Algorithm B produces higher values compared the ofate-of-the-art practical algorithms, and enjoys a lower variance in the quality of its output. However, due to the small values used for k, our algorithm requires significantly more queries to the objective function compared to the two other algorithms.
8698
Function value
86400
Funct
8478
8520
8353
0280
10
10
(a) Function values for the CIFARIO dataset.
(b) Function values for the CIFARIO0 dataset.
6008
127.8K
Algorichn
Ruchaiindr at ml. 2017 suchbinder et al. 2914
Funetion value
0489
Oracle calls
320.
8968
02.7H
0249
Aligar thee Buchbinder
55.
0129
1.0
1.0
k
(c) Function values for the Tiny ImageNet dataset.
(d) Number of queries for the CIFAR1O dataset.
237.9k
Algorithm Buchbindor ct al. 2817 Buchbinder et ni. 2014
197.9k 9K
Algorichm suchbinder et al. 2917 Buchbanderr et al. 2014
Oracle calls
calls
82, 7k
82, 7k
35.2k
55.2k
30
k
(e) Number of queries for the CIFAR100 dataset.
(f) Number of queries for the Tiny ImageNet dataset.
Figure 2: Experimental results for Personalized Image Summarization. Plots (a compare the output of our algorithm with the benchmark algorithms mentioned at the beginning of Section for a varying number of images. Each plot corresponds to different dataset. Plots dd) compare the number of queries used by the various algorithms.
3.3 Revenue maximization
Consider a company whose objective to promote product to users to boost revenue through the "word-of-mouth" effect. More specifically, given social network, we need to choose a subset of

---

up to & users to receive a product for free in exchange for advertising it to their network neighbors. and the goal is to choose users manner that maximizes revenue. The problem of optimizing this objective can be formalized as follows. The input is weighted undirected graph G = (V, , F) representing social network, where Way represents the weight of the edge between vertex and vertex j (with Waj = 0 if the edge (i.j) is absent from the graph). Given a set S S V of users who have become advocates for the product, the expected revenue generated is proportional to the total influence of S's users non-advocate users, formally expressed as (zz Lies Zievis Wij- It has been demonstrated that f' is non-monotone and submodular []]
Function value
13:47k
Funct sion value
2048
1.99
Algorithe
Buchbanndr at ml. 2017
4.59
Algaritho Eth
650
Buchbinder et al. 2014
Buctiaander #1 Md. 2017 Buclbssder
Function values for Advogato network dataset.
(b) Function values for Facebook network dataset.
665k
Algoritha
-uschennder 2017 -BchhSindrr et as. 2014
Oracle calls
230k
Grscle calls
2.300M
1.700M
266k
1.100M
133K
0.590N
(c) Number of queries for Advogato network dataset. (d) Number of queries for Facebook network dataset.
Figure 3: Experimental results for Revenue Maximization. Plots (a) and (b) compare the output of our algorithm with the benchmark algorithms mentioned the beginning of 3 for varying number & of images on the Advogato and Facebook network datasets. Plots (c) and (d) compare the number of queries used by the various algorithms.
In our experiment, compared the performance of Algorithm and the two benchmark algorithms on the Facebook network [42] and the Advogato network (29 The results of this experiment are depicted in Figure 3 Once again, algorithm enjoys both better output values and lower standard deviations compared to the benchmark algorithms. Our algorithm uses more queries compared to the Random Sampling algorithm of (6), but the ratio between the number of queries used by the two algorithms tends to decrease as k increases. The behavior of the Random Greedy algorithm (5) greatly depends k. For smaller values of & this algorithm requires roughly many queries as Random Sampling, but for larger value of & it requires significantly more queries than our algorithm.
4 Conclusion
In this work, we have presented novel algorithm for submodular maximization subject to cardinality constraint that combines a practical query complexity of O(n A-2) with an approximation guarantee of 0.385, which improves over the 1/eaaprrxximatio of the state-of-the-a practical algorithms. In addition to giving theoretical analysis of our algorithm, we have demonstrated empirical superiority (compared to practical state-of-the-art methods) in various machine learning applications. We hope future work will able improve the query complexity of our algorithm to cleanly linear without sacrificing either the approximation guarantee the practicality of the algorithm.
10

---

Acknowledgment
The work of Loay Mualem and Moran Feldman was supported in part by Israel Science Foundation (ISF) grant number 459/20.
References
[1] Eric Balkanski, Adam Breuer, and Yaron Singer. Non- mon-moottone submodular maximization in exponentially fewer iterations. Advances in Neural Information Processing Systems, 31, 2018 8.
[2] Eric Balkanski, Steven DiSilvio, and Alan Kuhnle. Submodular maximization in exactly queries. CoRR, abs/2406.00 00148, 2024.
[3] Niv Buchbinder and Moran Feldman. Constrained submodular maximization via nonsymmet- ric technique. Mathematics of Operations Research, 44(3):988-1055, 2019.
[4] Niv Buchbinder and Moran Feldman. Constrained submodular maximization via new bounds for dr-submodular functions. arXiv preprint arXiv:2311.01/29 2023.
[5] Niv Buchbinder, Moran Feldman, Joseph Naor, and Roy Schwartz. Submodular maximization with cardinality constraints. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, pages 1433-1452. SIAM, 2014.
[6] Niv Buchbinder, Moran Feldman, and Roy Schwartz. Comparing apples and oranges: Query trade-off in submodular maximization. Mathematics of Operations Research, 42(2):308-322,
2017.
[7] Niv Buchbinder, Moran Feldman, Joseph Seffi, and Roy Schwartz. A tight linear time (1/2)- approximation for unconstrained submodular maximization. SIAM Journal on Computing, 44(5):1344-1402, 2015.
[8] Gruia Cälineseu, Chandra Chekuri, Martin Pál, and Jan Vondrák. Maximizing a monotone submodular function subject matroid constraint. SIAM J. Comput., 406); 2011.
[9] Emmanuel J Candès and Benjamin Recht. Exact matrix completion via convex optimization. Foundations of Computational mathematics, 9(6):117-772 2009.
[10] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages (6650-9600) 2021.
[11] Chandra Chekuri, Jan Vondrák, and Rico Zenklusen. Submodular function maximization via the multilinear relaxation and contention resolution schemes. SIAM J. Comput., 43(6):1831-1899
2014.
[12] Yixin Chen, Ankur Nath, Chunli Peng, and Alan Kuhnle. Guided combinatorial algorithms for submodular maximization. arXiv preprint 2rXiv:2405.05202, 2024.
[13] Abhimanyu Das and David Kempe. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In ICML, pages 1057-1064, 201
[14] Ethan R. Elenberg, Alexandros G. Dimakis, Moran Feldman, and Amin Karbasi. Streaming weak submodularity: interpreting neural networks on the fly. In NeurlPS, pages 4047-4057, 2017.
[15] Alina Ene and Huy L Nguyen. Constrained submodular maximization: Beyond l/e. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), pages 248-257. IEFF, 2016.
[16] Uriel Feige, Vahab S. Mirrokni, and Jan Vondrák. Maximizing non-monotone submodular functions. SIAM J. Comput., 1334):1133-1153, 2011.

---

[17] Kai Han, Shuang Cui, Benwei Wu, et al. Deterministic approximation for submodular max- imization over a matroid in nearly linear time. Advances in Neural Information Processing Systems, 33:430-44) 2020.
[18] F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context, Acm Iransactions on inteructive intelligent systems (Tiss), 5(4): 2015.
[19] Charles R. Harris, K. Jarrod Millman, Stéfan J van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre -Mrrchan,,,,, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585:357-365 2020.
[20]
Chien-Chung Huang and Naonori Kakimura. Multi-pan pass streaming algorithms for monotone submodular function maximization. Theory of Computing Systems, 66( (1)(:3344394, 2022.
[21] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, Ontario, Canada, 2009.
[22] Alan Kuhnle. Quick streaming algorithms for maximization monotone submodular functions in linear time. In International Conference on Artificial Intelligence and Slutistics, pages 1360-1368. PMLR, 2021.
[23] Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. Numba: A llvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, pages
1 1-6, 2015.
[24]
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
[25] Jon Lee, Vahab S. Mirrokni, Viswanath Nagarajan, and Maxim Sviridenko. Non-monotone submodular maximization under matroid and knapsack constraints. In Michael Mitzenmacher, editor, Proceedings of the 41 Annual ACM Symposium on Theory of Computing (STOC), pages 323-332. ACM, 2009.
[26] Qi Lei, Lingfei Wu, Pin-Yu Chen, Alex Dimakis, Inderjit S. Dhillon, and Michael J. Witbrock. Discrete adversarial attacks and submodular optimization with applications to text classification. In Ameet Talwalkar, Virginia Smith, and Matei Zaharia, editors, MLSys. misys.org, 2019.
[27] Wenxin Li, Moran Feldman, Ehsan Kazemi, and Amin Karbasi. Submodular maximization in clean linear time. Advances in Neural Information Processing Systems, 05:17733-17887 2022.
[28] László Lovász. Submodular functions and convexity. In A. Bachem, M. Grötschel, and B. Korte, editors, Mathematical Programming: the State of the Art, pages 235-257. Springer, 1983.
[29] Paolo Massa, Martino Salvetti, and Danilo Tomasoni. Bowling alone and trust decline in social network sites. In IEEE International Conference on Dependable, Autonomic and Secure Computing (DASC), pages 658-663. IEEE Computer Society, 2009.
[30] Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, and Amin Karbasi. Fast constrained submodular maximization: Personalized data summarization. In ICML, pages 1358-1367.
PMLR, 2016
[31] Baharan Mirzasoleiman, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and An- dreas Krause. Lazier than lazy greedy. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1818- AAAI Press, 2015.
[32] Marko Mitrovic, Ehsan Kazemi, Morteza Zadimoghaddam, and Amin Karbasi. Data sum- marization at scale: A two-stage submodular approach. In ICML,, pages 3596-3605. PMLR, 2018.
[33] Loay Mualem and Moran Feldman. Using partial monotonicity in submodular maximization. Advances in Neural Information Processing Systems, 35:2723-2733 2022.
12

---

[34] Loay Mualem and Moran Feldman. Resolving the approximability of offline and online non- monotone dr-submodular maximization over general convex sets. In International Conference on Artificial Intelligence and Statistics, pages 2542-2564. PMLR, 2023.
[35] Loay Mualem, Murad Tukan, and Moran Fledman. Bridging the gap between general and down-closed convex sets in submodular maximization. arXiv preprint SKKt::2401.0925.1 2024.
[36] Loay Raed Mualem, Ethan Elenberg, Moran Feldman, and Amin Karbasi. Submodular mini- max optimization: Finding effective sets. In International Conference Artificial Intelligence and Statistics, pages 1081-1089 PMLR, 2024.
[37] George L Nemhauser and Laurence A Wolsey. Best algorithms for approximating the maximum of submodular set function. Mathematics of operations research, 3(3):177-188 1978.
[38] George I. Nemhauser, Laurence A Wolsey, and Marshall Fisher. An analysis of approximations for maximizing submodular set functions-i. Mathematical programming, 14:265-292 1978.
[39] Ashkan Norouzi-Fard, Jakub Tarnawski, Slobodan Mitrovic, Amir Zandich, Aidasadat Mousav- ifar, and Ola Svensson. Beyond 1/2-approximation for submodular maximization on massive data streams. In ICML, pages 3829-3838. PMLR, 2018.
[40] Benjamin Qi. On maximizing sums of non-monotone submodular and linear functions. In Sang Won Bae and Heejin Park, editors, International Symposium on Algorithms and Compula- tion (ISAAC), volume 248 of L/Ples, pages 41:1-41:16. Schloss Dagstuhl- Leibniz-Zentrum für Informatik, 2022.
[41] Murad Tukan, Fares Fares, Yotam Grufinkle, Ido Talmor, Loay Mualem, Vladimir Braverman, and Dan Feldman. Orbslam3-enhanced arXiv preprini arXiv:2312. 2...3312.33333 13385, 2023.
enhanced autonomous toy drones; Pioneering indoor exploration.
[42] Bimal Viswanath, Alan Mislove, Meeyoung Cha, and Krishna P Gummadi. On the evolution of user interaction facebook. Proceedings of the 2nd ACM workshop on Online social networks, pages 37 37-42, 2009.
[43] Jan Vondrák. Symmetry and approximability of submodular maximization problems. SIAM J. Comput., 42(1): 2013.
[44] T Wolf. Huggingface's transformers: Statc-of-the-art natural language processing. arXiv preprint 190.0711 2019.
[45] Lifeng Zhou and Pratap Tokekar. tion. IEEE Transactions on Robotics, 38(5):3064-3M)) -00044444444..1 2022.
submodular optimization for multirobot coordina-
13

---

A A warmup version of our algorithm
In this section, we present and analyze a simpler version of our algorithm with the same general structure, but excluding the speedup techniques used to obtain main result. Inspired by Buchbinder et al. [3, and similar to Algorithm 3] this simpler version (given as Algorithm 6 comprises three steps: (i) Searching for good initial solution that guarantees a constant approximation to the optimal set. This is accomplished by running the Twin Greedy algorithm of [17), (ii) Finding an (approximate) local search optimum set Z using a local search method. In this simple version, we use the classical local search algorithm for this purpose, which requires O(nk²) queries to the objective function. (iii) Lastly, we construct another solution using version of the Random Greedy algorithm suggested by Buchbinder et al. [5] that avoids elements of the set Z in its first iterations. The algorithm terminates by outputting the better of the two solutions generated in the last two steps.
A.1 Local search
In this section, we present (as Algorithm 4 simple local search algorithm, which is the algorithm
used to implement the first two steps Algorithm 6 Algorithm I begins by finding an initial solution
S using the Twin Greedy algorithm of [77. The algorithm then proceeds as follows: (i) If [|| < checks for an element и such that adding ur to S increases the function value by at least (1 + 26 (SS). If such an element is found, it is added to S. (ii) If [S] k. it looks for two elements u E N N and v E S such that swapping u and (i.e.. removing from S and adding u to S) increases the function value by least (1 + (S)). If such elements exist, the algorithm performs the swap. (iii) If no elements satisfy the previous two conditions, the algorithm checks for an element e such that removing u from increases the function value by (1 + 21 fS(). If such an element is found, is removed from S. Algorithm continues to search for elements satisfying any of the above three conditions until no such elements exist any longer. When this happens, the algorithm terminates and returns the set S as its output.
Algorithm 4: LOCAL-SEARCH((:.)
input : positive integer h 2 1, a non-negative submodular function f, and an error parameter E (0, 1).
output A set S C
Initialize S to be feasible solution guarantecing c-approximation for the problem for some constant c E (0, 1].
while true do
3u E ./ W such that f(S + u) 2 (1 f(5) and |S| < l then
S < 5 +
else if Bu E N S, E S such that f(S + v) 2 (1 + 2 f(S) and |S] k then
5 5 S 5-v
v u.
7
else if Eu E such that f(S - v) 2 (1+ S f(S) then
S 5 S u.
else
return S
The properties of Algorithm are formally established by Theorem A.1
Theorem A.1. Given positive integer k, non-negative submodular function f and an error parameter e E (0,1) Algorithm returns a set S C of size at most k such that
S(S) > F(SUOPT) + f(S OPT)
f(SnOFT)
and
2 E
1+6
while requiring Og (nk?) queries the objective function.
Proof. Let S denote the output of Algorithm and let E_, E+, and E. be defined as follows.
E__ is the event that there exists S such that f(S v) > (1 f(S).
14

---

E is the event that there exists E, 2, S such that f(S + a - v) 2 (1+ 2k f(S)
and [ k.
E+ is the event that there exists E WV such that S(S + u) 2 (1 25 ((S) and
|
Since the algorithm terminated with the set S, none of the events E4, F_ or E occurs. To lower bound the value of the set S, inspect the implications arising from each of these events not occurring.
Implications of E_ not occurring. Since E_ does not occur, for every E S, (1 + ((S) f(S - v). Summing this inequality across every element in E S CPT yields
(1 - 1(S) S(S) +
Σ
-)(8-v) [(S)]
S OPT
DES
2 f(S) +
(f(S OPT) f(5))
S OPT
where the second inequality holds by submodularity of f. By rearrangement, obtain that
S(S) Z
S OPT + ((SNOPT) 2 e+e/2 (SSOOPT)
(3)
26
where the last incquality holds since |S OPT < ||| S k.
The above proves the second inequality guaranteed by the theorem. Below we show that the first inequality guaranteed by the theorem is implied either by E+ not occurring, or by E+ not occuring. depending on the size of S.
Implications of E+ not occurring when [S] < k. Since E4 does not occur and |S| < k. for every W \ S, it holds that (1 + I(S) > S(S + u). Summing the above inequality across every element in OPT \ S yields
(1 + 2h
((S) 2 f(S)
1
Σ &EOPTYS
[S(S + u) S(S)
OPT S
2 f(S) + [OPT S) (f(SUOPT) - f(S))
where the second inequality follows from the submodularity of f. By rearrangement, now obtain
f(S) Z OPT S + S(S S(SUOPT)
1 1+e/2 ((SUOP
(4)
where the last inequality holds since JOPT |S S [PPT] < Averaging Inequalities (3) and gives
S(S) Z ((SUOPT) + f(S n OFT)
2 +
which proves the first inequality guaranteed by the theorem in the case of |S] k.
Implications of E+ not occurring when |S| = k. Since F, does not occur and |S] = k. for every pair u, where u E W' and S, (1 + & ((S) 2 S(S + - v). Summing the above inequality for every E S OPT and u E OFT \ S. yields that
(1
E 2h
S(S) 2
Σ
Σ
f(S -
S
OPTI
OPT
wEOPT 8 VES
15

---

Observe that
Σ Σ
u S(S)
SS
OPT 10PT S
#EOPT/S 88 vES)
B
Σ Σ
IfSS u f(S
Σ
E
[f(S - v) f(S)
SS
OES
aEOPT dES
[S P/OPT] OPT S
IS \ OPT 1OPT) \ S
To bound A, note that, by the submodularity of f,
Σ
u)
Σ IS(S + u) S(S)]
NEOPT
VEOPT
2 f(su - f(S),
and hence,
Σ E If(S u- f(S-((S - v)]
S DES
(SSUOPT) S(S) 1PPT \ S|
SS \ OPTI OPPT \ S)
To bound IB, we note that the submodularity of implies that
Σ E IS(S - v) - [(S)]
Σ
[S(S - - [(S)]
WEOFT S
DES
f(S n (OPPPP)--- - S(S)
2
SPOPTT
OPT) S
1S) (PTPPT
18 XPPT
Combining all of the above yields that
- ((S)
(ISUCPT) - S(S)
f(S)
2k
1S OPT
OPP \ S|
which by rearrangement implies
f(S n (PF)
F(S UPT)
f(S) >
+
S OPT
SYOETT
1+ 26
OPP [|++
CPTYST S/CPPT
26
TOPT/S
A(SUOTT) + f(S n CPT)
2 2+
where the last inequality holds since OOFT \ S = |S OPT < k. This completes the proof of the first inequality guaranteed by the theorem in the case of |S| = k.
To complete the proof of the theorem, it remains analyze the query complexity of Algorithm T First, we recall that S is initialized Line of Algorithm by the Twin Greedy algorithm of [17], which gives an approximation ratio e = 1/5 using O(n log k) queries to the objective function. Each iteration of the loop of Algorithm Acan can be implemented using O(nk) queries since there are only O(nk) ways to choose u E A and и S. Let us bound the number L. of such iterations. Observe that the function value of the set S in Algorithm 4 increases by multiplicative factor of 1 + 24 following each iteration the while loop (except for the last onc). Since is initialized with solution of value at least (OIPT) and its value is never larger than f(OPT) (because it remains feasible), we get
f(OPT) 2 (1+ 5) 9) cf(PP
and rearranging gives
In
In 5
+
Ok/e))
In(1
+
2k
E
In(1
+ 26
Combining the above results, we get that the query complexity Algorithm is upper bounded by O(nlog k) + ((k)) L = O(n log k) + O(nk) O(k/s) = O
D
16

---

A.2 Guided Random Greedy
In this section, we present the Guided Random Greedy algorithm (Algorithm 5). which is the variant of the Random Greedy algorithm of [5] used to implement the last step of Algorithm [ Algorithm S starts with an empty set, and adds one element in each iteration until returning the final solution after & iterations. In its first [k (, iterations, the algorithm ignores the elements of Z, and in the elements. However, except for this difference, the behavior of
rest of the iterations, it considers the algorithm in all iterations is very similar. Specifically, in each iteration i the algorithm does the following two steps. In Step (i) the algorithm finds subset M, of size i maximizing the sum of marginal gains of the elements E M; with respect to the current solution - In step (ii), the algorithm chooses a random element from M; and adds it to the solution. This algorithm implicitly assumes that [V] 2 3k. If this is not the case, one can fix that by adding to the ground set 2k dummy elements of value 0 before executing the algorithm (and then removing any dummy elements that appear in the solution of the algorithm).
Algorithm 5: Guided Random Greedy
input A set Z C N. positive integer k 2 1, non-negative submodular function f. and flip
point 14 E [0, 1]
output A set S C
Initialize Sg f- 0.
for i to [k te do
Let M, S U be a subset of size & maximizing ΣvEM, f(u S + u).
Let u, be a uniformly random element from Mj.
S; 5.+-S. S:-1
U4
for i = [A: 1. + to k do
Let M, NV S be a subset of size maximizing EvEM, f(ul 8 + u).
Let u, be a uniformly random element from Mj.
S, <- S Uz.
10 return St-
The properties of Algorithm are given by Theorem A.2
Theorem A.2. There exists an algorithm that given positive integer k. value i, E 10, non-negative submodular function f: 2N -> R2o, and a set S N obeying the inequalities given in Theorem A.1 outputs a solution Sk, obeying
k [t. k] ak [t, tak k- 16, a+ ((PP))
E[S(S)))
a tak-1_
2
2k -
ak (e
S(OPT (
k
a a (
where a = 1/k. Furthermore, this algorithm requires only O(nk) queries to the objective function.
To prove Theorem A.2 we first need to present some preliminaries. The Lovász extension of f is a function f: [0, 1 defined as follows. For every vector æ E [0, 1]
f(x) - (' f(Tx(z))dX
where (xxx) A {uEM I., A). The Lovász extension of submodular function is known to be convex. More important for us is the following known lemma regarding this extension. This lemma stems from an equality, proved by Lovász (28). between the Lovász extension of a submodular function and another extension known as the convex closure.
Lemma A.3. Let f: 2 + R be a submodular function, and let f be its Lovász extension. For every X E [0, 1/- and random set Dz C N obeying Pr|u E Dz] = En for every E NN (i.e., the marginals of Da agree with x), f(x) < Blf(D.)).
17

---

Using the last lemma, we now prove a lower bound on the expected value of the union of any set A with the solution S, of Algorithm P after i iterations.
Lemma A.4. For every integer 0 S i S k and set A CN, it holds that
Elf(s, A)]
}) 8,
{(A)
100011111
k ) (1 (1-1) 1) 1 1 S(AUZ).
8:
where B. = maxx (0, i [tg -kk)
Proof. Let x0) [0, 1|" be the vector of the marginal probabilities of elements belong to S In other words, for every element u E N, zu Pr[u E S:). Since each iteration of Algorithm adds each element to the solution with probability at most 1/k, the coordinates of x(() are all upper bounded by 1 (1 1/k)', For elements of Z we also know that they are not added by Algorithm to the solution the first [h tz] iterations, and therefore, their coordinates in (x()) are upper bounded by
1 - (1 1 /kk max -1 (1-1/k)1/))))
Let A denote vector in {0, 13 containing 1s at the entries that correspond to elements present in A and 0 in the remaining coordinates. We also denote by z²(*) V 1A the coordinate maximum of x00 and 1A- By Lemma A.3
Eff(s, U A)] 2 f(xs. V1A)= = / f(Tx(xs. VIa))a
1-
S(T2(xs, V IA))dA +
J(Tx(xs, V VIA)JA
1-(1-1)4
-11-1)--1
1-11-))-1
f(Tx(xs, V VIA))d++ + (1 (1-1)² S(A)
i-1
1)
2 (1--))-(1--)
(1- S(A) I(A))
(I(A)) - ((A))
where the second inequality holds by the non-negativity of f. and the last inequality follows since Tx (xs, V 1A) = Bx U A for some set BA C \ Z, and the submodularity and non-negativity of imply together that
f(Bx UA) f(A) UZ) f(AuZ) > f(A) f(Au
With the above result, we are now ready to bound the expected value of f(S)). Lemma A.5. Let = 1 r. Then, for every integer 0 < < [ta 11.
B[f(S.)] 2 (1 (1-o) a²) ( (/FF))Z) (1 (1-a² i(1 a)aa (iiD)s((PP ((OPTUZ)
and for every inleger .k] kkssikk
Eif(S.) 2 i [t kg
/ --tt, ((OPF) i i [t. 1e] (at- - f(OPT 2)
b + 1e.kk f((Stt.)))
Proof. Let E. be an event fixing all the random decisions in Algorithm 5 up to iteration i - 1 (including), and let A, = OPT for i < [t, k] and A, = OPT for i > It k7. Since all the elements of A, can be chosen to be in M,, and so the dummy elements, get that, conditioned on E,
[l 5.-1)) 2
max s(u 15-1)2= WEM.
Σ s(u) S NEM.
-ー Σ
f(u1S k (f U f(
wEA;
18

---

where the last inequality holds by the submodularity of f. Since S; = S + us rearranging the last inequality gives [[f().) 2 2 ((,-1 A,) + aS, Taking expectation now over all possible events Eg we get that, without conditioning on anything,
(5) Proving the first inequality of the lemma. We prove the first inequality of the lemma (for 0 s S [t. induction i. For i = 0 the inequality holds by the non-negativity of f. Assume now that 1 I i S it. k] and the inequality holds for i 1, and let us prove the inequality for i.
E[f(s.) 2 k EEf A,) + +aBif(S--i]]
[[f(s,)] 2 k - A.)] + aEif(S:-1))
x [[PPT 2) - (1 (1- (1- 1)²))) S(OPT 2)]
+a(1 a-- -f01 \2 2) al aa-a - -(1 (1-a)(i- (0)(i-1aa'33 f(OPT 2)
(1 (1-a)(((0P)) 2) - (1 a :(OPTU u2)
+ a1 - of 15- \ Z) a 11 1- (1 at(( - 1) S(OPT u 2)
= aa(1 - (o(1-a)][(OP]) \ 2) (1- a² (1 - (1-a)ia*)) a)iaa-1) /(OPUUZ)
where the second inequality holds by Lemma A.5 and the induction hypothesis since A, = OPT Z, and the first equality holds by definition of a.
Proving the second inequality of the lemma. We prove the second incquality of the lemma (for It, k] S i S k) by induction on i. One can verify that for i = [ts k] the inequality trivially holds. Assume now that [t. k1 < i S k and the inequality holds for i 1, and let us prove the inequality for i.
[f)( 2 k Ef A [ff(()
2 2
-(OPP)) - i- i-
-(11 -(OPT))(((1
1 i-
k
1 1(OPT u 2)] + i- 1-14s- [1] -
k
k
[to k1
a0' 2o -1 -1
- /(OPTU 2) + - ((S
k
k-
(a" [ta (OPP)) (aa a'=1) S(OPTU) UZ]
[1g k] - a' -1 (CPPT
k
[t. k3 - (a'- ',, -1
a MOPTU Z) az at ((S
k
a- x1 ((PPTT i i- [t. k] (d-- F(CPTUZ) 8 It. 13]
k
1 [t, f(S(r,,))
where the second inequality holds by Lemma A.5 and the induction hypothesis since A, OPT, and the first equality holds by definition of a.
We are now ready to prove Theorem A.2
Proof of Theorem A.2 Note that by submodularity and non-negativity of f.
SOOPT \ Z) > f(0) (OPP)) SOOP n Z) (CPFT)-((O))) n 2Z)
The lower bound in the theorem follows by combining the above inequality with the following two inequalities: the inequality arising by plugging = [t, A1 into the first case of Lemma A.5 and the inequality arising by plugging = le into the second case of Lemma A.5
To conclude the proof, observe that each one of the & iterations of Algorithm 5 requires us compute the marginal gain of most n elements with respect to the set St-1, which can be done using ((n) queries to the objective function per iteration, and (rak) queries in total.
19

---

A.3 0.385-Approximation Guarantee
We are now ready to present Algorithm (the simpler version of our main algorithm). Recall that this algorithm returns the better among the two sets produced in the last two steps described in the beginning of this appendix. Formally, these sets are the output sets of LOCAL--EARCCH and Algorithm
Algorithm 6: Warmup Algorithm
input :A positive integer k 2 1, non-negative submodular function f. and flip point 0 1 S 1
output A set S C
S1 LOCAI-SBARCH(:: f)- 2 S2 + the output of Algorithm
return (()
The following theorem is proved by setting in Algorithm to be small enough positive constant. Theorem A.6 (Approximation guarantee). Given an integer k 2 and a non-negative submodular function f: 2N Rzo, there exists a 1.355-aprrxiimation algorithm for the problem of finding set S C N of size at most maximizing f. This algorithm uses O(nk²) queries to the objective function.
Proof. The proof of the approximation guarantee is very similar to the corresponding part in the proof Theorem 2.3, and is thus, omitted. The query complexity stated in the theorem follows directly from Theorems A. and A.2 and the fact that is set to positive constant value.
B Omitted Proofs of Section 2
In this section, we prove the theorems whose proofs have been omitted from Section 2
namely,
Theorems 2.1 and 2.2
B.1 Proof of Theorem 2.1
In this section, we prove Theorem 2. We begin with the following lemma. We assume without loss of generality that OPT is of size k (otherwise, we add to OPT dummy elements).
Lemma B.1. If the set So provides c-approximation, then each iteration of the loop starting on Line in Algoritim 7 returns a set with probability at least k/(ce(1 L). Moreover, when this happens, the output set S returned obeys
f(Sn CPT) + f(SU
and S(S) > f(S
S(S)
2+
1+6
Proof. For every two integers 0 <i< L and 1 < [log 1 we denote by A the event that the set S obeys the condition on Line 13 of Algorithm it holds that
i.e., the event that for every integer 0 < t < k
Σ f(ul 8/)
min SCS; wes
Σ 1 u) -) +
WES
To better understand the implication of an event A assume that such event occurs, and observe that for = JPPT) \ s/), holds that
K
S(S!) S(S n OPPT) +cf(S)) =
Σ
flue JÍ U (w.....,u-1)) + +
(6)
|
>
Σ
flue Jz. LK -ee + +
E=1
20

---

Σ f(e) -) + =f((s)
/OPT
min SC8, vES
[(e S v) e((s))
E S
max SCA
WES
Σ #EOPT(S
fu (()S)) [( U UPTT) ((S))
where the first equality holds by setting J! a S/NOPT,K/ & S SPPT and using u1, u2..... [K] to denote the elements of K? in some arbitrary order. The first and last inequalities follow from submodularity of f. the second inequality holds since the fact that S/ and OPT are both size implies that t = JOFT \ S = |s/ \ OPT), and the third inequality holds under the assumption that event A/ occurs. Rearranging the last inequality, we get
f(S)) S(S) U OOPT) S(S) n NOPT)
2+
In addition, the existence of the dummy elements implies that plugging this inequality into Inequality (6)
max SC WES
Σ f(u ( 2 0, and
yields that the event Af also implies
f(S)) - f(S) f(S) n OFTT) + ef()) N
min E1 - -)) +ff(s)) 20,
SCS/.1S mt DES
and rearranging this inequality gives f(8))
nCSTCC 1tc
The above shows that to prove the lemma it suffices to show that the probability that A holds is at least (c(cc(( L)) Towards this goal, let us study the implications of the complementary event A² Specifically, would like to lower bound TE [((L_)) f(S)) A?]
Fix particular set S that causes the event A to occur (notice that the occurrence of this event depends only on the set 5²). Then, there must exist sets T+ C \ S and T_ S S of size S such that
Σ f(u S > Σ f(e 8 v) + = TET
VET
We can now define the event By as the event that Z24 OT+ + Ø (notice that the event B is defined only for this particular set SJ). The probability of the event B is
Pr(B/ 5 > 1 ("
n1 -//- [T+
2 (1-14/ 1/0) T+
2 1
where the first inequality holds since (1 - 1 < - for any > 0, and the second inequality
holds for any E [0, 1] by the concavity of 1 - - By the law of total expectation and the fact that f(S is always at least f(S)), we get
1 ((((StL)) f(S)) sis Pr(B/ ()) EE ((S{ f(8)) 1 s B(]
[T+]
2 (1 1/-)
1 1/4 (OFT))
21

---

where the last inequality holds since
E - { fZ - f(S)) |s/ By
f((((((()))
1(5 - + +(4) [(5) +4+ - m(41 - ((5)) S/ By]
E -
- - Da
E [(S + +w) f((() |5 By +B f(5 { -() f((()) Sq. Bi
Σ f(u S MET
Σ f(v) 8 u) PET
cf(S)) [4+ z
CES(OPT)
T+
T-
IT+
where the second inequality holds by submodularity of f(-). the third inequality holds by the way Algorithm | chooses u² and uf if we let u' be a uniformly random element of T+ 7 Z and r' be uniformly random element of T_, the penultimate inequality holds by our assumption that S implies the event A/, and finally, the last incquality holds since it is guaranteed that f(S)) 2 f(8)) = f(o)) f(OPT).
Since the above bound on the expectation holds conditioned on every set S that implies the event A/. it holds (by the law of total expectation) also conditioned on the event A itself. Adding this lower bound for all values, and using the non-negativity of f, we get
E [[((S)]
2
Σ k (1
Σ
Pr(A)) E ([((()L)) ((S))
-(()) A!]
f=
L
1- 1/2 (O((T)))
Σ Pr((A))
Combining the last inequality with the fact that f(S))) is deterministically at most S(OPT)) it must hold that
1 1- 1/e k: cs
PPr(A) (A:
Σ
L
AP
S
k
CE (1 1/c)
Hence, the probability that the event A². does not hold for a uniformly random i* [L]
E' Pr
A
L
CE(1 - 1/4
Theorem 2.1. There exists an algorithm that given a positive integer k, a value & E (0, 1), and non-negative submodular function f: 2N - Rzo, oulputs set S SCC. N of size at max ls that, with probability at least E, obeys
f(S) N f(Sn OPT) + +f(SUOF) and f(S) F(SNOPT)
2+
1+s
Furthermore, the query complexity of the above algorithum is Oz(n + k2).
Proof. As mentioned above, we initialize the set So using the deterministic 1/4-approximation algorithm of Balkanski et al. [2], which uses only O(n) queries to the objective function. Thus,
c 1/4 in our implementation of Algorithm Let us now set
c=(1-14] Algorithm
24
Then, Lemma B. guarantees that every iteration of the outer loop of the algorithm returns a set (obeying the requirement of the theorem) with probability at least 1/2. Hence, by repeating this loop [logg~ times, we are guaranteed that Algorithm] outputs set with probability at least
1 - c. To complete the proof of the theorem, only remains to bound the number of queries to
the objective function that are necessary for implementing it. Each iteration of Algorithm can be
22

---

implemented using OOn/k k + k) queries, and for the above choices of L, Algorithm 1 has only Oz iterations. Thus, all the iterations of the algorithm can be implemented using O + k2) queries in total. It should also be mentioned that evaluating the condition on Line 13 of the algorithm requires O(n + k) queries the objective, and since this condition is evaluated log E - (1(1) times, all its evaluations require in total only Og + k)) queries.
B.2 Proof of Theorem 2.2
We begin by observing that, like Algorithm [ each iteration of Algorithm 2 adds each element u E N into the solution with probability at most 1/k, and furthermore, the first [t. iterations of the algorithm do not pick elements of all (see the analysis of Sample Greedy in [6] for proof of similar observation that is given in more detail). Given this observation, the proof of Lemma applles also to Algorithm 2 Thus, this lemma can be used in the proof of the following result.
In this section, we prove Theorem 2.2
Lemma B.2. Let a =
1 k.-
Then, for every integer 0 sii [tx R.).
B[f(s.) 2 (1 - at) f(OPT) 2) - (1 a² - i(1 a aaaaa) F(OFTU 2)
2ci
k
and every integer [1g k] Sisk
[t k]
a+-1) k a k1 f(OP)) i [t. 1e] (at -1--1-aa111 (f F(DFT Z)
E[f(s.) 2
h
2: [ta.kk
+ 1 ke [(S
k
Proof. Let Fr be an event fixing all the random decisions in Algorithm 2 up to iteration i
(including), and let A; = OPT Z for i I [t,- k] and A; = CPT for i > [he k]. Since all the elements of A, can be sampled in iteration i, by following the proof of Lemma 13 in the analysis of the Sample Greedy algorithm by [6). one can obtain that, conditioned on E,
1 -
E[max((0, (0, [(us ()-1)))
Z
To be more specific, Lemma of 6 shows that with probability at least E the element chosen as in iteration i of Algorithm 2 belongs to the k elements with the largest marginal values among the elements that can be sampled in this iteration (if less than elements can be sampled, dummy elements should added for the purpose of this argument). Let B, denote the set of these k elements. Since the probability of each element of B, to be selected as u, is non-decreasing f(u, (S-1). by Chebyshev's 's sum inequality, we get
[[max f(u S,-1) 2 (1 -1- EUEB, max(0, s(u S b:
EmEA, (1 Su Si-1) 1-E -
(1- (1--)
-
k
where the second inequality holds since B, contains the elements with the largest marginals among the elements that can be samples, and A, is a set of up to k such elements; and the last inequality follows from the submodularity f.
Since S. S + u, when f(u, S.-1) > 0 and S, = S otherwise, we get f(S)) - ((,-1) = max(0, (0, f(u S.,1)). Plugging this observation into the previous inequality, and rearranging gives
1 -
k f(S:-1 + (1- 1 - 5(Si-1)
E[f(S)) 2
Σ + [({-- + af(;;-1) 8
f(S;-1UA)
UA) + (f
f(OPT)
where the second inequality uses the non-nigatyvity of f, and the last inequality holds since f(S,-1 U A,) < f(S.11) + f(A)) f(S A,) I f(S.-1) + f(A)) < 2f(FPT) because both S and
23

---

A, are feasible solutions, and thus, cannot have a value larger than (QPT)) Taking expectation now over all possible events E; we get that, without conditioning on anything,
28 [[(().) 2 k - aB[[(S
l: S(OPT)
The remaining part of this proof omitted since it is very similar to the corresponding part in the proof of Lemma A.5 except that the last inequality should be used instead of Inequality (5).
We are now ready to prove Theorem 2.2
Theorem 2.2. There exists an algorithm that given a positive integer k, a value E (0, 1). a value 10, a non-negative submodular function f: 2 + R20, and set Z C N' obeying the inequalities stated in Theorem 2.1 outputs a solution Sy, obeying
[[[(5)))
k
[1. k.
k-Ta a k-1 [6, a+) 1(0P))))))+++
a + tak 2k [t. k. F (e- S 2)
+ (a an S n Z) 22 (CPP)
where a - 1/k. Moreover, this algorithm requires only (z(n) queries to the objective function.
Proof of Theorem 2.2 The lower bound in Theorem 2.2 follows from the arguments used in the proof of Theorem
A.2 except that Lemma B.2 is used instead of Lemma A.5 which results the
additional error term 2 (OP)) in the lower bound of Theorem 2.2
To bound the number queries to the objective function necessary for implementing Algorithm 2 observe that each iteration of Algorithm Sashhmm@@aaan samples Oz (n(n/k) elements, and the marginal gain (with re- spect to S,-1) has to be computed only for the sampled elements. Thus, each iteration of Algorithm requires only Oz (n/k) queries to the objective function. Since the algorithm has only k iterations, its total query complexity is k (n(nkk (n)n).
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our theoretical contribution can be found in Section 2 while the empirical evaluation our method can be found in Section 3
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [NA]
Justification: Our paper has no limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

The proofs of the theoretical results can be found in Section and Appendix
Justification:
Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All the information needed to reproduce the main experimental results of the paper can be found in Section 3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
26

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code can accessed at the URL http:://ggittbb..c//m/raaaaaaaaa
G85ApproximationSS oaSukaa
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
Coaee//dddeSebmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the details be found in Section 3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We present the error bars as the shaded regions in of the figures. In Section clearly explain that the sizes of the error bars determined by the standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
27

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative еттоr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: In Section we wrote the computer type and memory needed to reproduce the experiments. Due to high dependence on implementation details, we do not provide the time of execution. Instead, we provide in Section the number of queries needed to obtain the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/neurips.. http:////ierpppp.cp/b/bbttttttit
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdictio)..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper studies a well-known submodular optimization problem without societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
28

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The creators original owners of assets (e.g., code, data, models) used in the paper are properly credited in Section 3
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
29

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper involves neither crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper involves neither crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
30

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "Yf5g2ACNAs", "reviewer": "Reviewer_USjH", "summary": "The paper presents a practical $0.385$-approximation algorithm using $O(n+k^2)$ queries for non-monotone submodular maximization under a cardinality (size) constraint, where $n$ is the number of elements in the ground set and $k$ is the maximum size of a feasible solution. As a comparison, the state-of-the-art algorithms attain a $0.401$ approximation ratio with a high number of queries, or $1/e-\\epsilon$ approximation ratio using $O_{\\epsilon}(n)$ queries. The paper also evaluates its method experimentally on several applications.", "questions": "1.\tI’m curious about if it’s possible to discretize the algorithm in [3] to get a practical $0.401$ algorithm, just like this paper.\n2.\tFollowing are some typos:\n1) I suggest using “at most” to replace “at max” in the paper.\n2) Page 13. “Let S be a subset in N of size at (missing most) k”.\n3) Page 16, Lemma A.3, $x_{\\in}$ --> $x\\in$.\n4) Page 16, in the last inequality, \\beta^{i} should be \\beta_{i}.\n5) Page 17, in the first formula, A\\cup B_{\\lambda}\\cap Z should be A\\cup B_{\\lambda} ‘\\cup’ Z.\n6) Page 18, Lemma B.1, “that starts returns a set…” doesn’t read smoothly.\n7) Page 18, in the first sentence of the proof of Lemma B.1, S_{i,j} should be S_i^j.\n8) Page 20, in inequality (6), the subscript L should be i.", "limitations": "Yes", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper makes a substantial progress for the studied problem by presenting a new algorithm with low query complexity and improved approximation ratio. The algorithm is obtained by carefully combining several well-known techniques and methods in the field. Such a result already contains enough originality as a NeurIPS submission.\n2.\tThe paper is complete, in the sense that it contains both a correct theoretical analysis and detailed experimental evaluations of its algorithm.\n3.\tThe paper is organized well. It splits the algorithm into several ingredients and then explains each part in detail and how to combine them to get the final result. In such a way, it’s easy for readers to follow the basic ideas and check the correctness of the algorithm.\n4.\tThe paper, as suggested by the experiments, provides a better solution for several important applications. On the other hand, it further exploits several mature techniques in the field, which may inspire more results in the area.", "weaknesses": "1.\tThere are many typos in the proofs (see Questions for some examples), which affects the clarity. The authors are suggested to proofread their paper in a later version.\n2.\tThe main algorithmic framework is borrowed from [2]. So, if the space is enough, I suggest the authors to add a paragraph discussing why the framework can beat the $1/e$ ratio. In doing so, the paper might be more readable to those who haven’t read [2] before.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate the reviewer's meticulous evaluation of our paper. Thank you for your invaluable feedback. In what follows, we address the reviewer’s questions/comments.  We will be happy to engage with you to address any lingering concerns\n\n  1. **There are many typos in the proofs (see Questions for some examples), which affects the clarity. The authors are suggested to proofread their paper in a later version.**\n    \n\nWe have addressed the typos raised in the Questions section and will thoroughly pass over the paper to fix any additional types. We thank you for your keen observation. \n\n  2. **The main algorithmic framework is borrowed from [2]. So, if the space is enough, I suggest the authors to add a paragraph discussing why the framework can beat the $\\frac{1}{e}$ ratio. In doing so, the paper might be more readable to those who haven’t read [2] before.**\n\n\nWe thank the reviewer for the fruitful suggestion. We will be more than happy to add such a paragraph to the paper.\n\n\n  3. **I’m curious about if it’s possible to discretize the algorithm in [3] to get a practical $0.401$ algorithm, just like this paper.**\n\nMaking the recent 0.401-approximation of [3] practical is an interesting question that we have also considered. Unfortunately, this goal seems to be very challenging and requires significant new ideas because the algorithm of [3] has a query complexity that is exponential in 1/epsilon. Notice that this complexity remains exponential even if we assume access to the multilinear extension of the objective function rather than requiring the algorithm to use samples to estimate it. Making the algorithm combinatorial, as we have done in the current paper, can make this assumption justified, but it does not help to alleviate the inherent exponentiality of the query complexity of the other parts of the algorithm.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FrWHDhy3Gt", "reviewer": "Reviewer_q8Tb", "summary": "This work addresses the problem of maximizing a non-monotone submodular function subject to a cardinality constraint. In submodular maximization, we have a set of elements (ground set) and a function that assigns a value to any subset of elements. A function is submodular if adding an element to a smaller set contributes more value than adding it to a larger set. Formally, for sets A and B where A is a subset of B, adding an element x to A increases the function value more than adding x to B. A function is monotone if its value always increases with set size. Non-monotone functions don't necessarily have this property. The goal here is to find a subset of size at most k from the ground set that maximizes the value of the non-monotone submodular function.\n\nWhile they developed a 0.385-approximate solution for this problem, there is an algorithm with a better approximation factor of 0.401. Another work also achieves a 0.385-approximate solution, but both are impractical due to the high number of query calls required. In contrast, this work uses only O(n+k^2) query calls. Additionally, there are two works with a 0.367-approximation, one requiring O(nk) query calls and the other O(n) query calls. In the experiment section, the authors compared the output value of their algorithm with those of the two latter algorithms.\n\nThe core idea of the proposed algorithm involves:\n- Running the sample greedy algorithm by Buchbinder et al. [5].\n- Applying a local search technique to improve the initial solution.\n- Employing a Stochastic greedy method and returning the solution with the highest value between the local search and Stochastic greedy outputs.\n\nThis approach leverages existing techniques while incorporating additional steps to enhance the solution within a reasonable computational cost.", "questions": "- What is the query complexity of your algorithm in terms of epsilon? \n- Do you have any practical comparisons of the number of query calls used in your experiments?", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "For non-monotone submodular maximization under a cardinality constraint, their work achieves the best query complexity for algorithms that attain at least a 0.385-approximate solution. In other words, among practical algorithms—those that can be implemented and executed in a reasonable time—they offer the highest approximation factor.", "weaknesses": "Since their algorithm does not have the best approximation factor, it is less interesting for theoretical work and more important for practical applications. Therefore, a comprehensive experiment supporting their claims is crucial. While they did a great job by running experiments on different problems and demonstrating the advantage of their output value compared to two other algorithms with worse approximation guarantees, it is also necessary to plot the number of query calls for these algorithms. Comparing the number of query calls in addition to the output value is essential, as they claimed to have the best approximation algorithm with \"a low and practical\" query complexity.\n\n### Comments for the authors:\n- Line 43: Remove “over”\n- Line 50: I think [14] present a ½-approximation algorithm for symmetric submodular functions and for non-symmetric, they only achieve a ⅖-approximation algorithm.\n- Line 68: Do you mean “3 applications”?\n- Line 149: Remove “)” after “pn”\n- Line 208: Replace “she is” with “they are”\n- Line 548: Replace “lemmata” with “lemmas”\nThe revenue maximization experiment is mostly known as the max-cut problem. I think it is good to at least mention that.\n- In Algorithm 3, the term “AIDED-MEASURED DISCRETE STOCHASTIC GREEDY” has been used to refer to Algorithm 2. However, the name of Algorithm 2 is different. \n- From lines 95 and 96, I assumed that Algorithm 2 improves the output of Algorithm 1. However, after reading the algorithms, it seems that Algorithm 2 finds a different", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for the reviewer’s positive evaluation of our work. In what follows, we address the concerns raised by the reviewer in detail. We will be happy to engage to address any lingering concerns.\n\n  1. **Comparing the number of query calls in addition to the output value is essential, as they claimed to have the best approximation algorithm with \"a low and practical\" query complexity. | Do you have any practical comparisons of the number of query calls used in your experiments?**\n\nWe thank the reviewer for the insightful suggestion. We have generated graphs depicting the number of query calls, comparing the efficiency of our method with that of our competitors. These graphs can be found in the PDF file of the general rebuttal.\nInterestingly, the number of queries used by our algorithm is empirically dominated by the queries used for the initialization of the Fast Local search procedure (Line 1 of Algorithm 1). Implementing this initialization in the way described in the submitted paper leads to a query complexity that is somewhat worse than that of other algorithms. However, an alternative implementation of this initialization is implied by a very recent paper [1] that describes a fast ¼-approximation algorithm for maximizing non-monotone submodular functions under matroid constraints (of which cardinality constraints are a special case). When initialized this way, our algorithm becomes much faster (in terms of the empirical number of query calls), while the quality of the solution produced in terms of the objective value remains almost unchanged. We stress that the change in the initialization method only affects the implementation of Line 1 of Algorithm 1 in our paper.\n\n\n\t[1] Balkanski, Eric, Steven DiSilvio, and Alan Kuhnle. \"Submodular Maximization in Exactly n Queries\" arXiv preprint arXiv:2406.00148 (2024).\n\n\n  2. **What is the query complexity of your algorithm in terms of epsilon?**\n\nThe query complexity of our algorithm is $O\\left(n * \\epsilon^{-2} \\log\\left(\\frac{1}{\\varepsilon}\\right) + k^2 * \\varepsilon^{-1} \\log\\frac{1}{\\varepsilon}\\right)$, excluding the query complexity required for obtaining the initial solution $S_0$ for the Fast Local search procedure. The query complexity required for getting this initial solution is independent of epsilon when the algorithm of [1] is used for that purpose, as described in the response to the previous question.\n\n\n  3. **From lines 95 and 96, I assumed that Algorithm 2 improves the output of Algorithm 1. However, after reading the algorithms, it seems that Algorithm 2 finds a different**\n\n\nIntuitively, Algorithms 1 and 2 balance each other. In a sense, Algorithm 2 uses the output of Algorithm 1 as a “set to avoid”, and its analysis shows that when the output of Algorithm 1 has a poor value, then by avoiding it Algorithm 2 is guaranteed to output a set of good value. Thus, it is guaranteed that the better of the outputs of the two algorithms is always a good solution.\n\n  4. **Typos/writing suggestions/comments**\n\nWe have addressed all of the raised comments and suggestions concerning the writing. We are thankful for the keen observation of the reviewer and the fruitful suggestions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y9MlTDkdeW", "reviewer": "Reviewer_yvAe", "summary": "The paper introduces a new algorithm, FAST-LOCAL-SEARCH, for maximizing non-monotone submodular functions, achieving a 0.385-approximation with low query complexity. It combines initial solution search, accelerated local search, and stochastic greedy improvement steps to outperform existing algorithms in machine-learning applications like movie recommendation, image summarization, and revenue maximization. The algorithm guarantees a constant approximation to the optimal set and is supported by theoretical proofs and empirical evaluations.", "questions": "Please refer to weaknesses.", "limitations": "N.A.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper introduces an algorithm, FAST-LOCAL-SEARCH, for maximizing non-monotone submodular functions under a cardinality constraint. This algorithm combines initial solution search, accelerated local search, and stochastic greedy improvement steps to achieve a 0.385-approximation with query complexity of O(n + k^2). The practical performance of the algorithm has been validated in real-world applications.", "weaknesses": "The novelty of the paper is limited. The idea of this paper is derived from existing work [2], and more significantly, the \"guided algorithm\" proposed in this paper is very similar to [10] in both algorithm design and theoretical analysis.\n\nThe authors claim that the algorithm proposed in this paper has a practical query complexity of O(n+k^2); however, in practical applications, the value of k is often very large, making an O(k^2) query complexity potentially impractical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the detailed review. We hope that our response below addresses all their concerns about the paper. If further clarification is needed, we will be happy to provide it.\n\n  1. **The novelty of the paper is limited. The idea of this paper is derived from existing work [2], and more significantly, the \"guided algorithm\" proposed in this paper is very similar to [10] in both algorithm design and theoretical analysis.**\n\nAs mentioned by the reviewer, our algorithm is based on ideas from [2], and its main novelty is in finding a way to implement these ideas efficiently and practically. As explained in Section 1.1, the work of [10] is an independent parallel work that is also based on [2]. Both works come up with a very similar basic algorithm having a query complexity of O(nk). However, the two works diverge beyond this point. The objective of [10] was to derandomize this basic algorithm and extend it to other constraints. In contrast, our objective was to further speed up the algorithm beyond the above-mentioned query complexity. This additional speed-up is an important technical contribution of our work. For example, getting the local search part of the algorithm of [2] to run fast (as implemented in Algorithm 1) required us to relax the notion of local optimum used and introduce a sophisticated probabilistic analysis deviating from the well-threaded path of analysis of local search algorithms in submodular optimization.\n\n\n  2. **The authors claim that the algorithm proposed in this paper has a practical query complexity of O(n+k^2); however, in practical applications, the value of k is often very large, making an O(k^2) query complexity potentially impractical.**\n\n   As mentioned by the reviewer, the query complexity of our algorithm is O(n + k^2), which is linear for k = O(n^{½}). For applications that require larger values of k, our algorithm indeed runs in super-linear time, which is unfortunate. Still, one should note two things:\n   - The PDF of the general rebuttal includes a plot comparing the empirical query complexity of our algorithm with various existing algorithms, including linear query complexity algorithms like the one of [5]. This plot demonstrates that our algorithm compares very well with these algorithms in terms of the empirical query complexity (while outcompeting them in terms of the value of the solution produced).\n\n   - The only other algorithm in the literature that guarantees an approximation ratio better than 1/e and has a possibly practical query complexity is the algorithm due to the recent independent work of [10]. This algorithm has a query complexity of $O(nk)$, which is worse than the query complexity of our algorithm except in the regime of k = $\\Omega(n)$ (and in this regime the query complexities of both algorithms become identical).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Uh3crdjhyQ", "reviewer": "Reviewer_2jpf", "summary": "The paper studies the problem of approximately maximizing a submodular function under a constraint that the sets can be of size at most $k$. By carefully combining algorithms from previous work and developing them further, the authors obtain an algorithm that guarantees a $0.385$-approximation with $O_\\epsilon(n + k^2)$ queries to the function and a user-defined probability of failure. The good performance of the proposed algorithm is demostrated with several experiments.", "questions": "In terms of the running time, i.e., not the number of queries, how does the proposed algorithm compare against the state of the art? Because the running time depends on, for example, $epsilon$, it is not immediately clear to me how large the constant factors of the algorithms would be.\n\nOther:\n- Abstract: Without previous background on the topic, it was unclear to what $k$ and $n$ referred to in the abstract.\n- Lines 29–32: first, $B$ is a subset of $A$, but then $A$ is a subset of $B$. This can be confusing to the reader.\n- Line 66: If I understood the paper correctly, I think it would be more precise to use $O_\\epsilon$ instead of the $O$-notation here (although a constant $\\epsilon$ is used in the experiments)\n- Caption of Alg. 3: maximiziation -> maximization\n- Line 172: OPT is not written with \\mathbb", "limitations": "Since the paper focuses on a optimizing a method that is used in ML applications and does not study application directly, I think the limited discussion of limitations is sufficient here.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Even though the work relies heavily on previous work as described in the beginning of Sec. 3, their combination appears highly nontrivial to me, and the techniques have been developed further from the previous work.\n\nThe empirical results demonstrate clear improvements over previous state of the art and appear to have significantly less variance in the output.", "weaknesses": "Because of my limited expertise on the topic, my largest complaint is on the presentation of the results: Although the authors do a good job in explaining the ideas behind the math, the formulas are occasionally relatively cumbersome to read, e.g., Line 14 of Algorithm 1. On the other hand, I must admit that I do not have an immediate solution on how to improve them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive evaluation of our work. We have thoroughly addressed each of the reviewer’s raised concerns and are eager to engage in further discussion to ensure all remaining issues are resolved.\n\n  1. **In terms of the running time, i.e., not the number of queries, how does the proposed algorithm compare against the state of the art? Because the running time depends on, for example, $epsilon$, it is not immediately clear to me how large the constant factors of the algorithms would be.**\n\nThe runtime is a problematic measure as it highly depends on implementation details and the level of optimization employed, often making comparisons based on this method non-robust. Thus, it is customary in the literature about submodular maximization to use the number of queries to the objective function as a more reliable proxy for the real-world performance of an algorithm. Accordingly, we include in the PDF of the general rebuttal a plot comparing the number of queries used by our algorithm and central algorithms from the literature.\nInterestingly, the number of queries used by our algorithm is empirically dominated by the queries used for the initialization of the Fast Local search procedure (Line 1 of Algorithm 1). Implementing this initialization in the way described in the submitted paper leads to a query complexity that is somewhat worse than that of other algorithms. However, an alternative implementation of this initialization is implied by a very recent paper [1] that describes a fast ¼-approximation algorithm for maximizing non-monotone submodular functions under matroid constraints (of which cardinality constraints are a special case). When initialized this way, our algorithm becomes much faster (in terms of the empirical number of query calls), while the quality of the solution produced in terms of the objective value remains almost unchanged. We stress that the change in the initialization method only affects the implementation of Line 1 of Algorithm 1 in our paper.\n\n\n\t[1] Balkanski, Eric, Steven DiSilvio, and Alan Kuhnle. \"Submodular Maximization in Exactly n Queries\" arXiv preprint arXiv:2406.00148 (2024).\n\n\n  2. **Typos/Writing suggestions/comments**\n\nWe thank the reviewer for their keen observations. We have addressed your comments and revised the manuscript accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate the reviewers for their constructive feedback and insightful questions about our research. Your dedication and expertise in helping us enhance our work is invaluable. We are grateful for your observations and thank you for your contributions.
Finally, attached here is our PDF containing all the figures depicting our algorithm with a different initialization in comparison with our algorithm with the original initialization discussed in the appendix and also in comparison with the presented competitors in the paper. We present in these figures a comparison with respect to two metrics: 
  - Objective value, and
  - Oracle calls


The initialization we are referring to is from [1] and is referred to as "Algorithm 1 with different initialization" in the uploaded graphs.

    [1] Balkanski, Eric, Steven DiSilvio, and Alan Kuhnle. "Submodular Maximization in Exactly n Queries" arXiv preprint arXiv:2406.00148 (2024).

Meta-review: The ratings for this paper are {7, 6, 5, 6}. Moreover, Reviewer USjH is willing to champion the paper (score 7).

**Remark 1.** I checked DBLP and Reviewer USjH has not published a paper with any authors of this submission (arXiv:2405.13994).

**Decision.** I think we should accept this paper on the grounds that all reviewers lean accept.

**Remark 2.** That said, I read the paper myself prior to the reviews and was hoping it would be simpler and cleaner than it ended up being (since it doesn't advance the SotA approximation ratio). My personal rating would be 4 (lean reject).
Decision: Accept (poster)