Constrained Robust Submodular Partitioning

1 Introduction . The problem of partitioning a given set V of items into m blocks , where any two blocks share no items in common , arises in many real-world scenarios and machine learning applications . As an optimization problem , partitioning aims to generate the blocks so that the utilities of the blocks , as measured by a given set function , are good . Submodular functions are a rich family of set functions that naturally captures diversity of a given set of items . They have been applied in many realworld problems [ 26 , 30 , 21 , 17 , 13 , 18 , 28 ] . By maximizing a submodular utility function for each partitioned block , we encourage each block to be representative of the ground set V . Many algorithms have been proposed for various submodular partitioning problems with approximation guarantees . For the submodular welfare problem [ 24 ] , we aim to find a partition such that the sum of the submodular evaluations of every block is maximized . Such an objective promotes the overall utility of the entire partition but some blocks may still have small function values . The robust submodular partitioning problem [ 12 , 29 ] ( often called “ submodular fair allocation with indivisible goods ” ) aims to find the partition such that the minimum-valued block in the partition is maximized according to the submodular function . The robust objective optimizes the worst block in the partition so that all blocks are minimally “ good. ” In the general setting , every block in the partition may have a different submodular function ( the heterogeneous case ) although for this work , we study only the restricted setting where all blocks share the same submodular function ( the homogeneous case ) . The robust submodular partitioning problem has many applications . Given V as the training dataset for a machine learning task , Wei et al . [ 29 ] finds a partition of V for distributed training : every block of partitioned data is sent to a single machine for parallel gradient computations , and the gradients 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . are aggregated over all the blocks in the partition for model updates . Since we enforce each block to be representative of V , the gradients computed across distributed machines are consistent , resulting in reduced variance and improved convergence for the aggregation step . Using a similar idea , Wang et al . [ 25 ] partitions the training data into mini-batches so that every mini-batch is as representative as possible , therefore reducing the variance during mini-batch gradient-based training . In this work , we explore two different algorithmic approaches , Min-Block Greedy and Round-Robin Greedy , for our partitioning problem but under various constraints , newly applied to this problem . For Min-Block Greedy based algorithms , we first show that the 1 m bound for the unconstrained case is tight . We then modify the algorithm to allow a general down-closed constraint C , and prove an approximation bound of ↵ ↵m+1 , where ↵ is the bound for solving the submodular maximization problem under constraint C using a greedy based algorithm . For example , for a cardinality constraint , ↵ = 1 1/e [ 9 ] , and the bound for constrained robust submodular partitioning is ( m+ 11 1/e ) 1 . Similarly , for C as an intersection-of-p-matroids constraint , ↵ = ( 1 + p ) 1 [ 10 ] , and the bound is ( m+ p+ 1 ) 1 ; for C as a knapsack constraint , ↵ = 0.5 ( 1 1/e ) [ 16 ] , and the bound is ( m+ 21 1/e ) 1 . For Round-Robin Greedy based algorithms , when C is a cardinality constraint , we get a bound of ( 1 1/e ) 2 3 , and when C is a matroid constraint , we get a bound of 1 1/e5 . The Min-Block Greedy approach gives a weaker bound , and since the 1 m bound for the unconstrained case is tight , we can not improve upon the 1 m factor for the constrained case . The Round-Robin Greedy approach gives a constant bound , but its running time is worse . The running time for Min-Block Greedy is O ( n2 ) , where n is the ground set size . For RoundRobin Greedy under a matroid constraint , the running time is O ( n2 ( log logm+log 1 ) ) , as it needs to use binary search to find the optimal solution value to the given problem over an exponentially decreasing sequence , with 11+ ( > 0 ) as the multiplicative factor . In all cases , we assume an oracle model , and the running time is in terms of the number of submodular evaluations . An important contribution our work shows is that by utilizing the Min-Block Greedy algorithm result as input , our Round-Robin Greedy algorithm attains strongly polynomial running time — all previous results on the unconstrained case using a Round-Robin-like algorithm have only weakly polynomial running time [ 3 ] . The various constraints ( e.g. , cardinality , matroids , and knapsack ) we study greatly improves the applicability of robust submodular partitioning . Several applications that benefit from the constraints include : ( 1 ) Partition a training data for machine learning models in distributed training or forming deterministic mini-batches [ 29 , 25 ] . The additional constraint can be the number of samples from each class to be no more than a certain value . If there are enough samples in the training data , every resulting block will have the same number of samples for each class , which avoids imbalance , further promoting each block ’ s diversity , and improving the gradients ’ consistency . ( 2 ) Given an undirected graph , we partition the edges into subgraphs so that each subgraph is representative based on the submodular evaluation , and we also constrain each subgraph to have no cycles ( a cycle matroid ) . A practical scenario is that we wish to send information efficiently over a graph of devices . We partition the graph so that information can be sent in parallel , and the constraint to have no cycles enforces that information is not redundantly sent twice to the same device , leading to improved communications efficiency . ( 3 ) Again , for an undirected and connected graph , we partition the edges into subgraph blocks such that if we were to remove any block of the partition from the original graph , the remaining graph remains connected ( which can be done via a bond matroid , where min-cuts are cycles and anything not a cut is independent ) . In practice , this functions as a form of reliability insurance . For a graph of devices , we partition the graph to perform computation in parallel , so that if the connections in one block fail , the other blocks can still operate and communicate since the graph remains connected . 2 Related Work . Golovin [ 12 ] introduces robust submodular partitioning ( i.e. , submodular fair allocation of indivisible goods ) , and proposes a matching-based algorithm with a bound of 1 n m+1 . Khot & Ponnuswami [ 14 ] proposes a binary search based algorithm and gives an improved bound of 12m 1 . Asadpour & Saberi [ 2 ] uses an ellipsoid approximation approach and gives a bound of ⌦ ( 1p nm1/4 logn log3/2 m ) . Wei et al . [ 29 ] gives a simple Min-Block Greedy algorithm and proves a 1 m bound . A Round-Robin Greedy approach is given in [ 3 ] with a bound of 1 e 1 3 . Ghodsi et al . [ 11 ] proposes a local search algorithm with a bound of 13 . Both [ 3 ] and [ 11 ] requires guessing of the optimal solution value from an exponentially decreasing sequence of values , so strictly speaking , they lose an extra ( 1+ ) -factor in the approxima- tion bound where ( 1 + ) is the exponential factor for the guessing sequence . We can set the value small to get close to the constant bounds shown above at the costs of computation . Wang et al . [ 25 ] extends the Min-Block Greedy algorithm with a cardinality constraint , and also shows a hierarchical partitioning framework to reduce the memory costs . We adapt the Min-Block Greedy approach [ 29 ] and the Round-Robin Greedy approach [ 3 ] to the constrained case . To the best of our knowledge , this work is the first ( as far as we know ) to study the robust submodular partitioning problem under all of the various constraints ( cardinality , intersection of matroid , knapsack ) . Wang et al . [ 25 ] is a special case of our work as it only studies the cardinality constraint . Cotter et al . [ 8 ] studies ( as well as allowing multiple blocks to be jointly scored ) a matroid constrained “ groupings ” ( i.e. , coverings , packings , or partitions ) problem but only a fractional subset of groups ( rather than the minimum of the groups ) , is guaranteed to have values larger than the bounded max-min OPT , while our bound compares the min block evaluation to the optimal max-min value . Another line of related research is the submodular load balancing problem , which minimizes the maximum-valued block in the partition according to the submodular evaluations . In contrast to promoting diversity of each block for the robust submodular partition problem , submodular load balancing encourages every block to contain redundant items , similar to standard clustering objectives . Theoretically , this problem has been shown to be much harder as Svitkina & Fleischer [ 22 ] shows a information theoretical lower bound of o ( q n logn ) , and also gives a sampling algorithm to match the lower bound up to constant factors . Similar to the max-min case , Ghodsi et al . [ 11 ] uses the ellipsoidal approximation to get a bound of O ( p n log n ) . Wei et al . [ 29 ] gives a Lovász extension based relaxation algorithm and achieves a bound of m . 3 Preliminaries and Formulation . With a ground set V of n items , a submodular function f is a set function 2V ! R that satisfies the property : f ( A ) + f ( B ) f ( A [ B ) + f ( A \ B ) , where A , B ✓ V . Equivalently , a submodular function is characterized by diminishing returns : f ( v|A ) f ( v|B ) 8v /2 B and A ⇢ B ✓ V , where f ( v|B ) = f ( { v } [ B ) f ( B ) . Submodular functions naturally describe the diversity or representativeness of a given set of items . Many simple greedy-based algorithms have been developed to solve optimization problems involving submodular functions , giving both theoretical approximation guarantees , as well as good empirical performance . We restrict the submodular functions discussed in this paper to be monotone non-decreasing and normalized , i.e. , f ( B ) f ( A ) 8A ✓ B ✓ V , f ( ; ) = 0 . A matroid M = ( V , I ) is a set system that describes the independence relationships among the subsets of the ground set V . I is a set of subsets of V and every S 2 I is considered an independent subset . The matroid rank function is defined as rM ( A ) = max { |S| : S ✓ A , S 2 I } . rM ( V ) indicates the maximum size of a subset that may be independent according to the matroid M. All subsets of cardinality k with some integer k > 0 form a uniform matroid , which we denote by Mu k . A partition matroid is one where we partition the ground set into blocks , and a set is independent if it intersects each block by no more than a block-specific limit . We define a particularly useful partition matroid on an expanded ground set V̄ as follows : We first duplicate the ground set m times , creating V1 = V2 = . . . Vm = V , which are ground set copies . We create an expanded ground set V̄ = ] j=1 : mVj as the disjoint union . A subset S ✓ V̄ is independent in Mpm if for every element v 2 V , let its m copies in V̄ be { v1 , v2 , . . . , vm } , we have |S \ { v1 , v2 , . . . , vm } | 1 , i.e. , S contains at most one copy of element v. Apart from the uniform matroid and this particular partition matroid , there are many other matroids reflecting a natural notion of independence , for example , the linearly-independent set of real vectors and the spanning trees in a graph . In the below , we use both S 2 M and , when clear , S 2 I , to indicate that S is independent in the matroid M = ( V , I ) . Matroids are often used as constraints in submodular optimization problems : maxS2I f ( S ) with a matroid M = ( V , I ) . When M is a uniform matroid Mu k , this reduces to the cardinality submodular max and the greedy algorithm gives a 1 e 1 bound [ 9 ] . For a general constraint with the intersection of p matroids , the simple greedy algorithm gives a 1 p+1 bound [ 10 ] . Suppose we represent a set S as a binary indicator vector xS 2 { 0 , 1 } n , i.e. , 8i 2 [ n ] , xS [ i ] = 1 if vi 2 S or otherwise xS [ i ] = 0 . Then for all the independent sets of a matroid M = ( V , I ) , the convex hull over all the xS , S 2 I forms a polytope , which is called the matroid polytope PM of matroid M [ 9 ] . Based on the convex property of the matroid polytope , algorithms [ 4–6 , 24 ] have been proposed to firstly solve a continuous extension of the submodular optimization problem under the matroid polytope constraint , which generates a fractional solution in [ 0 , 1 ] n , and then round the fractional solution to an integral solution to get the resulting set . The continuous greedy algorithm [ 4 ] gives a 1 e 1 guarantee under a single matroid constraint using pipage rounding [ 1 , 4 ] . Interestingly , running the continuous greedy under a partition matroid constraint ( submodular welfare problem ) gives a uniform fractional solution , i.e. , on the expanded ground set V̄ , the fractional solution x = ( 1 m , 1 m , . . . , 1 m ) ( i.e. , assigning 1 m of every element to each block ) leads to a 1 e 1 bound in expectation for an integral solution that assigns each element in V uniformly to one of the m blocks . This observation also constitutes the basic idea of Round-Robin Greedy for solving the robust submodular partition problem [ 3 ] , which we will discuss in more detail later . For a submodular function f on a ground set V , the robust submodular partition problem ( submodular fair allocation ) [ 12 ] is defined as : max ⇡2⇧ ( V , m ) min A2⇡ f ( A ) , ( 1 ) where m is the number of blocks in a partition , we denote all possible partitions with m blocks of ground set V as ⇧ ( V , m ) , and one partition ⇡ with |⇡| = m is a collection of m disjoint sets . Equivalently , we can represent the partition using a partition matroid constraint on the expanded ground set V̄ : max S✓V̄ , S2Mpm min j2 [ m ] f ( S \ Vj ) . ( 2 ) Intuitively , the above optimization for robust submodular partitioning encourages the minimumvalued block to have a high submodular evaluation . Compared to the submodular welfare problem , the robust submodular partition promotes fairness for every one of the partition blocks . There have been three recent approximation algorithms developed to solve Eq . ( 1 ) . Particularly , Wei et al . [ 29 ] uses a Min-Block Greedy algorithm , which greedily adds the element with the largest gain to the block with the minimum evaluation . Barman & Krishna Murthy [ 3 ] propose a Round-Robin Greedy algorithm , which iteratively traverses all the blocks in a fixed order , and greedily adds an element with the largest gain to each block . Ghodsi et al . [ 11 ] applies a local search approach , which starts with an arbitrary partition and keeps moving an element from a non-minimum block to the minimum block if this relocation improves the objective by certain threshold until no such element can be found . For [ 3 ] and [ 11 ] , they both require guessing the optimal solution ’ s value , and they need to run multiple instances of their algorithms with the guessed optimal values as an exponentially decreasing sequence from the maximal possible value f ( V ) to the optimal solution value µ = max⇡2⇧ ( V , m ) minS2⇡ f ( S ) . With the exponential decreasing factor as 1 + , the running time ( in terms of submodular function calls ) is O ( n2 1 log f ( V ) µ ) for [ 3 ] , and O ( n2m2 1 log f ( V ) µ ) for [ 11 ] . Min-Block Greedy , a much simpler algorithm , has a running time of O ( n2 ) . Note that the settings of [ 3 ] and [ 11 ] are slightly more general than Eq . ( 1 ) as the submodular function for each block can be different . But it ’ s not the heterogeneous case either as they focus on a different notion of optimality .

Summary: The focus of this work is constraint robust submodular maximization. In this problem, the goal is to find $m$ sets maximizing the minimum value of a given submodular function $f$ over these sets. This problem has attracted attention recently and  algorithms are designed for offline and streaming setting. In this work, author consider this problem subject to three famous constraint and provide two type of algorithms.  1- Min-block greedy algorithms: This algorithm has been used before for non-constraint version robust submodular maximization and provides a $1/m$ approximate solution.  In this work they extend it to the constrained version and achieve $\frac{\alpha}{\alpha m+1}$ approximation algorithm, where $\alpha$ is the approximation guarantee of the greedy offline algorithm for that constraint. Intuitively this result is very close to being tight with respect to the unconstrained for large $m$ values.  2- Round Robin greedy algorithm: They present a $\frac{(1-e^{-1})^2}{3}$-approximation algorithm for the cardinality constraint and  $\frac{1-e^{-1}}{5}$ for the matroid constraint.