Cardinality constrained submodular maximization for random streams

We consider the problem of maximizing submodular functions in single-pass streaming and secretaries-with-shortlists models , both with random arrival order . For cardinality constrained monotone functions , Agrawal , Shadravan , and Stein [ ASS19 ] gave a single-pass ( 1 − 1/e − ε ) -approximation algorithm using only linear memory , but their exponential dependence on ε makes it impractical even for ε = 0.1 . We simplify both the algorithm and the analysis , obtaining an exponential improvement in the ε-dependence ( in particular , O ( k/ε ) memory ) . Extending these techniques , we also give a simple ( 1/e − ε ) -approximation for non-monotone functions in O ( k/ε ) memory . For the monotone case , we also give a corresponding unconditional hardness barrier of 1 − 1/e + ε for single-pass algorithms in randomly ordered streams , even assuming unlimited computation . Finally , we show that the algorithms are simple to implement and work well on real world datasets . 1 Introduction . Over the past few decades , submodularity has become recognized as a useful property occurring in a wide variety of discrete optimization problems . Submodular functions model the property of diminishing returns , whereby the gain in a utility function decreases as the set of items considered increases . This property occurs naturally in machine learning , information retrieval , and influence maximization , to name a few ( see [ IKBA20 ] and the references within ) . In many settings , the data is not available in a random-access model ; either for external reasons ( customers arriving online ) or because of the massive amount of data . When the data becomes too big to store in memory , we look for streaming algorithms that pass ( once ) through the data , and efficiently decide for each element whether to discard it or keep it in a small buffer in memory . In this work , we consider algorithms that process elements arriving in a random order . Note that the classical greedy algorithm which iteratively adds the best element [ NWF78 ] can not be used here , and hence we must look for new algorithmic techniques . The motivation for considering random element arrival comes from the prevalence of submodularity in big-data applications , in which data is often logged in batches that can be modelled as random samples from an underlying 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . distribution.1 The problem of streaming submodular maximization has recently received significant attention both for random arrival order and the more pessimistic worst-case order arrival [ ASS19 , AEF+20 , AF19 , BMKK14 , BV14 , CGQ15 , FKK18 , FNSZ20 , HKMY20 , IV19 , KMZ+19 , MV19 , NTM+18 , HTW20 , Sha20 ] . 1.1 Submodular functions and the streaming model . Let f : 2E → R+ be a non-negative set function satisfying f ( S∪ { e } ) −f ( S ) ≥ f ( T ∪ { e } ) −f ( T ) for all S ⊆ T ⊆ E\ { e } . Such a function is called submodular . For simplicity , we assume f ( ∅ ) = 0.2 We use the shorthand f ( e|S ) : = f ( S ∪ { e } ) − f ( S ) to denote the marginal of e on top of S. When f ( S ) ≤ f ( T ) for all S ⊆ T , f is called monotone . We consider the following optimization problem : OPT : = max |S|≤k f ( S ) where k is a cardinality constraint for the solution size . Our focus on submodular maximization in the streaming setting . In this setting , an algorithm is given a single pass over a dataset in a streaming fashion , where the stream is a some permutation of the input dataset and each element is seen once . The stream is in random order when the permutation is uniformly random . When there are no constraints on the stream order , we call the stream adversarial . At each step of the stream , our algorithm is allowed to maintain a buffer of input elements . When a new element is streamed in , the algorithm can choose to add the element to its buffer . To be as general as possible , we assume an oracle model — that is , we assume there is an oracle that returns the value of f ( S ) for any set S. The decision of the algorithm to add an element is based only on queries to the oracle on subsets of the buffer elements . The algorithm may also choose to throw away buffered elements at any given time . The goal in the streaming model is to minimize memory use , and the complexity of the algorithm is the maximum number of input elements the algorithm stores in the buffer at any given time . For the oracle model , an important distinction is between weak oracle access or and strong oracle access . In the weak oracle setting , the algorithm is only allowed to query sets of feasible elements ( sets that have cardinality less than k ) . In the strong oracle setting however , the algorithm is allowed to query any set of elements . All our results apply to both the weak and strong oracle models . Our aim will be to develop algorithms that only make one pass over the data stream , using Õε ( k ) memory , where k is the maximum size of a solution set in I . We assume that k is small relative to n : = |E| , the size of the ground set . 1.2 Our contributions . On the algorithmic side , we give ( 1−1/e−ε ) and ( 1/e−ε ) approximation for cardinality constrained monotone and non-monotone submodular maximization respectively , both using O ( k/ε ) memory . The monotone result has an exponential improvement in memory requirements compared to Agrawal et al . [ ASS19 ] ( in terms of the dependence on ε ) , while the non-monotone result is the first to appear in the random-order streaming model , and improves upon the best known polynomialtime approximations under adversarial orders [ AEF+20 ] . The algorithms are extremely simple to implement , and perform well on real world data ( see Section 5 ) , even compared to offline greedy algorithms . On the hardness side , we prove that a ( 1 − 1/e + ε ) -approximation for monotone submodular maximization would require Ω ( n ) memory ( even with unlimited queries and computational power ) . This improves the hardness bound of 7/8 from [ ASS19 ] . 1.3 Related work . Prior work on this problem has focused on both the adversarial and random-order streaming setting . Algorithmic and hardness results further depend on whether the function f is monotone or non- 1Note that the random order assumption generalizes the typical assumption of i.i.d . sampling stream elements from a known distribution , as the random order assumption applies equally well to unknown distributions . 2Submodular functions f with f ( ∅ ) > 0 only improves the approximation ratio of our algorithms . monotone , and whether f has explicit structure ( e.g . such as by presenting a set system for f in the coverage case ) , or accessible only via oracle queries . Table 1 describes all the relevant results . Algorithmic results . Submodular maximization in the streaming setting was first considered by Badanidiyuru et al . [ BMKK14 ] who gave a ( 1/2− ) -approximation in O ( k log k/ ) memory for monotone submodular functions under a cardinality constraint , using a thresholding idea with parallel enumeration of possible thresholds . This work led to a number of subsequent developments with the current best being a ( 1/2− ) -approximation in O ( k/ ) memory [ KMZ+19 ] . It turns out that the factor of 1/2 is the best possible in the adversarial setting ( with a weak oracle ) , but an improvement is possible in the random order model ( the input is ordered by a uniformly random permutation ) . This was first shown by Norouzi-Fard et al . [ NTM+18 ] , who proved that the 1/2-hardness barrier for cardinality constraints can be broken , exhibiting a ( 1/2 + c ) -approximation in O ( k/ε log k ) memory where c < 10−13 . In a breakthrough work , Agrawal , Shadravan , and Stein [ ASS19 ] gave a ( 1 − 1/e − ε ) -approximation using k 2poly ( 1/ε ) memory and running time . We note that this is arbitrarily close to the optimal factor of 1 − 1/e , but the algorithm is not practical , due to its dependence on ε ( even for = 0.1 , the resulting constants are astronomical ) . Lower bounds . A few lower bounds are known for monotone functions in the adversarial order model : with a weak oracle , any ( 1/2 + ) -approximation would require Ω ( n/k ) memory [ NTM+18 ] . Under a strong oracle , a lower bound of Ω ( n/k3 ) memory for any ( 1/2+ ) -approximation algorithm was shown in a recent paper by Feldman et al . [ FNSZ20 ] . Another recent lower bound was proved by McGregor and Vu [ MV19 ] : a ( 1− 1/e+ ε ) -approximation for coverage functions requires Ω ( n/k2 ) memory ( this lower bound holds for explicitly given inputs , via communication complexity ; we note that this is incomparable to the computational ( 1− 1/e+ ε ) -hardness of maximum coverage [ Fei98 ] ) . For non-monotone functions , Alaluf et al . [ AEF+20 ] proved an Ω ( n ) memory lower bound for the adversarial order model with unbounded computation . In the random-order model , Agrawal et al . [ ASS19 ] show that beating 7/8 ( for monotone submodular functions ) requires Ω ( n ) memory . In contrast , we show that same construction as McGregor and Vu [ MV19 ] also applies to randomly ordered streams : ( 1− 1/e+ ε ) for coverage functions requires Ω ( n/k2 ) memory even in the random-order model . Submodular maximization in related models . A closely related model is the secretary with shortlists model [ ASS19 ] , where an algorithm is allowed to store a shortlist of more than k items ( where k is the cardinality constraint ) . Unlike the streaming model however , once an element goes into the shortlist , it can not be removed . Then , after seeing the entire stream , the algorithm chooses a subset of size k from the shortlist and returns that to the user . We note that the algorithms developed in this paper apply almost without modification to the shortlists model . 1.4 Overview of our techniques . Main algorithmic techniques . The primary impetus for our algorithmic work was an effort to avoid the extensive enumeration involved in the algorithm of Agrawal et al . [ ASS19 ] which leads to memory requirements exponential in poly ( 1/ε ) . To make things concrete , let us consider the input divided into disjoint windows of consecutive elements . The windows containing actual optimal elements play a special role — let ’ s call them active windows — these are the windows where we make quantifiable progress . When the stream is randomly ordered , we would ideally like to have each new element sampled independently and uniformly from the input . This leads to the intuition that the optimal elements are evenly spread out through all the windows . This can not be literally true , since conditioned on the history of the stream , some elements have already appeared and can not appear again . However , a key idea of Agrawal et al . [ ASS19 ] allows us to circumvent this by reinserting the elements that we have already seen and that played a role in the selection process . What needs to be proved is that elements that were not selected can still appear in the future , conditioned on the history of the algorithm ; that turns out to be true , provided that our algorithm operates in a certain greedy-like manner . To ensure progress was made regardless of the positioning of the optimal elements , previous work made use of exponentially large enumerations to essentially guess which windows the optimal elements arrive in . Where we depart from previous work is the way we build our solution . The idea is to use an evolving family of solutions which are updated in parallel , so that we obtain a quantifiable gain regardless of where the optimal elements arrived . Specifically , we grow k solutions in parallel , where solution Li has cardinality i . In each window , we attempt to extend a collection of solutions Li ( for varying i ) by a new element e ; if e is beneficial on average to every Li in the collection , we replace each Li+1 with the new solution Li ∪ { e } . Regardless of which windows happen to be active , we will show that the average gain over our evolving collection of solutions is analogous to the greedy algorithm . This is the basis of the analysis that leads to a factor of 1− 1/e . In addition to our candidate solutions Li , we maintain a pool of elements H that our algorithm has ever included in some candidate solution . We then use H to reintroduce elements artificially back into the input ; this makes it possible to assume that every input element still appears in a future window with the same probability , which is key to the probabilistic analysis leading to 1− 1/e . Non-monotone functions . Our algorithm for non-monotone submodular functions is similar , with the caveat that here we also have to be careful about not including any element in the solution with large probability . This is an important aspect of the randomized greedy algorithm for ( offline ) non-monotone submodular maximization [ BFNS14 ] which randomly includes in each step one of the top k elements in terms of marginal values . We achieve a similar property by choosing the top element from the current window and a random subset of the pool of prior elements H . Hardness results . Our hardness instances have the following general structure : there is a special subset of k good elements , and the remaining n − k elements are bad . The good elements are indistinguishable from each other , and ditto for the bad elements . In the monotone case , any k bad elements are a ( 1 − 1/e ) -factor worse than the optimal solution ( k good elements ) . Suppose furthermore that for parameter r > 0 , as long as we never query the function on a subset with ≥ r good elements , the good elements are indistinguishable from bad elements . The only way to collect ≥ r good elements in the memory buffer is by chance – until we ’ ve collected the required number of good elements , they are indistinguishable from bad elements , so the subset in the memory buffer is random . The classic work of Nemhauser and Wolsey [ NW78 ] constructs a pathological monotone submodular function with r = Ω ( k ) , which we use to prove that without memory Ω ( n ) the algorithm can not beat 1− 1/e . McGregor and Vu [ MV19 ] construct a simple example of a coverage function with r = 2 , which we use for our Ω ( n/k2 ) bound . For an exponential-size ground set , we extend their construction to r = 3 which translates to the improved lower bound of Ω ( n/k3/2 ) .

Summary: This paper considers cardinality constrained submodular maximization, both monotone and non-monotone, in the random streaming setting. In the random streaming setting the universe is received in a uniformly random permutation, as opposed to the adversarial setting where the universe could be received in any order.  In the random streaming setting, better approximation guarantees are possible compared to the adversarial setting (where 1/2 is the best possible). An algorithm developed by [ASS19] was shown to give a $1-1/e-\epsilon$ guarantee for cardinality constrained monotone submodular maximization, but has memory exponential in $\epsilon$ making it impractical. This paper introduces a streaming algorithm that gives the $1-1/e-\epsilon$ needing only $O(k/\epsilon)$ memory. They also prove $1-1/e$ is the best possible. Further, they introduce a streaming algorithm for the non-monotone version of the problem that gives a $1/e$ approximation guarantee also in $O(k/\epsilon)$ memory.  The algorithms are analyzed empirically, and shown to be practical.