Title: Faster Discrete Convex Function Minimization with Predictions: The M-Convex Case
Venue: NeurIPS 2023
Accepted: yes
Keywords: algorithms with predictions

beyond the worst-case analysis of algorithms

time complexity

combinatorial optimization

discrete convex analysis

submodular functions
Abstract: Recent years have seen a growing interest in accelerating optimization algorithms with machine-learned predictions. Sakaue and Oki (NeurIPS 2022) have developed a general framework that warm-starts the *L-convex function minimization* method with predictions, revealing the idea's usefulness for various discrete optimization problems. In this paper, we present a framework for using predictions to accelerate *M-convex function minimization*, thus complementing previous research and extending the range of discrete optimization algorithms that can benefit from predictions. Our framework is particularly effective for an important subclass called *laminar convex minimization*, which appears in many operations research applications. Our methods can improve time complexity bounds upon the best worst-case results by using predictions and even have potential to go beyond a lower-bound result.

Faster Discrete Convex Function Minimization with Predictions: The M-Convex Case

Abstract

Recent years have seen a growing interest in accelerating optimization algorithms with machine-learned predictions. Sakaue and Oki (NeurIPS 2022) have developed a general framework that warm-starts the L-convex function minimization method with predictions, revealing the idea's usefulness for various discrete optimization problems. In this paper, we present a framework for using predictions to accelerate M-convex function minimization , thus complementing previous research and extending the range of discrete optimization algorithms that can benefit from predictions. Our framework is particularly effective for an important subclass called laminar convex minimization , which appears in many operations research applications. Our methods can improve time complexity bounds upon the best worst-case results by using predictions and even have potential to go beyond a lower-bound result.

1 Introduction

Recent research on algorithms with predictions [29] has demonstrated that we can improve algorithms' performance beyond the limitations of the worst-case analysis using predictions learned from past data. In particular, a surge of interest has been given to research on using predictions to improve the time complexity of algorithms, which we refer to as warm-starts with predictions for convenience. Since Dinitz et al. [11]'s seminal work on speeding up the Hungarian method for weighted bipartite matching with predictions, researchers have extended this idea to algorithms for various problems [7, 35, 10]. Sakaue and Oki [39] have found similarities between the idea and standard warm-starts in continuous convex optimization and extended it to L-convex function minimization , a broad class of discrete optimization problems studied in discrete convex analysis [31]. They thus have shown that warm-starts with predictions can improve the time complexity of algorithms for various discrete optimization problems, including weighted bipartite matching and weighted matroid intersection.

In this paper, we extend the idea of warm-starts with predictions to a new direction called M-convex function minimization , another important problem class studied in discrete convex analysis. The Mconvexity is known to be in conjugate relation to the L-convexity. Hence, exploring the applicability of warm-starts with predictions to M-convex function minimization is crucial to broaden further the range of algorithms that can benefit from predictions, as is also mentioned in [39]. This paper mainly discusses an important subclass of M-convex function minimization called laminar convex minimization ( Laminar ), a large problem class widely studied in operations research (see references in Section 1.2). To make it easy to imagine, we describe the most basic form ( Box ) of Laminar ,

where f 1 , . . . , f n : R → R are univariate convex functions, R ∈ Z , ℓ 1 , . . . , ℓ n ∈ Z ∪ {-∞} , and u 1 , . . . , u n ∈ Z ∪ { + ∞} . Note that the variable x ∈ Z n is an integer vector, which is needed when,

for example, considering allocating R indivisible resources to n entities. As detailed later, adding some constraints and objectives to Box yields more general classes, Nested and Laminar , where the level of generality increases in this order. Streamlining repetitive solving of such problems by using predictions can provide substantial benefits of saving computation costs, as we often encounter those problems in, e.g., resource allocation [22], equilibrium analysis [16], and portfolio management [8].

1.1 Our contribution

We give a framework for accelerating M-convex minimization with predictions building on previous research [11, 39] (Section 3). We show that, given a vector ˆ x ∈ R n that predicts an optimal solution x ∗ ∈ Z n , the greedy algorithm (Algorithm 1) for M-convex function minimization finds an optimal solution in O( T init + T loc ∥ x ∗ -ˆ x ∥ 1 ) time, where T init and T loc represent the time for converting ˆ x into an initial feasible solution and for finding a locally steepest descent direction, respectively. Since we can minimize ∥ x ∗ -ˆ x ∥ 1 provably and approximately given optimal solutions to past instances [11, 23], this framework can provide better time complexity bounds benefiting from predictions. We also discuss how to apply our framework to general M-convex function minimization in Section 3.1.

Section 4 presents our main technical results. We apply our framework to Laminar , Nested , and Box and obtain time complexity bounds shown in Table 1. Our time complexity bounds can improve the existing worst-case bounds given accurate predictions. In particular, our O( n ∥ x ∗ -ˆ x ∥ 1 ) -time bound for Laminar improves the existing O( n 2 log n log mR n ) -time bound even if prediction error ∥ x ∗ -ˆ x ∥ 1 is as large as O( n ) . Our result for Nested is a corollary of that for Laminar and improves the existing worst-case bound if ∥ x ∗ -ˆ x ∥ 1 = O(1) . In the case of Box , we can further reduce the time complexity to O( n +log n · ∥ x ∗ -ˆ x ∥ 1 ) by modifying the algorithm for Laminar . Notably, our algorithm for Box runs in O( n ) time if ∥ x ∗ -ˆ x ∥ 1 = O( n/ log n ) , even though there exists an Ω( n log log( R/n 2 )) -time lower bound for Box [19]. As far as we know, this is the first result that can go beyond the lower bound on the time complexity in the context of warm-starts with predictions. Experiments in Section 5 confirm that using predictions can improve empirical computation costs.

Few studies in the literature have made explicit improvements upon the theoretically fastest algorithms, even if predictions are accurate enough. The only exception is [7], whose shortest-path algorithm with predictions can improve the best worst-case time complexity by a couple of log factors. By contrast, our methods with accurate predictions can improve the best worst-case bounds by O( n ) (up to log factors) in the Laminar case and potentially go beyond the lower-bound result in the Box case. Thus, our work not only extends the class of problems that we can efficiently solve using predictions but also represents a crucial step toward revealing the full potential of warm-starts with predictions. In this paper, we do not discuss the worst-case time complexity of our algorithms since we can upper bound it by executing standard algorithms with worst-case guarantees in parallel, as discussed in [39].

1.2 Related work

Algorithms with predictions [29], improving algorithms' performance by using predictions learned from past data, is a promising subfield in beyond the worst-case analysis of algorithms [38]. While this idea initially gained attention to improve competitive ratios of online algorithms [36, 2, 28, 1],

recent years have seen a surge of interest in improving algorithms' running time [11, 7, 39, 35, 10]. A comprehensive list of papers on algorithms with predictions is available at [27]. The most relevant to our work is [39], in which predictions are used to accelerate L-/L ♮ -convex function minimization, a large problem class including weighted bipartite matching and weighted matroid intersection. On the other hand, warm-starts with predictions remain to be studied for M-convex function minimization, 2 another essential class that is in conjugate relation to L-convex function minimization in discrete convex analysis [31]. Although our basic idea for utilizing predictions is analogous to the previous approach [11, 39], our algorithmic techniques to obtain the time complexity bounds in Table 1 for the specific M-convex function minimization problems are entirely different (see Section 4).

M-convex function minimization includes many important nonlinear integer programming problems, including Laminar , Nested , and Box , which have been extensively studied in the context of resource allocation [22]. A survey of recent results is given in [41]. Table 1 summarizes the worst-case time complexity bounds relevant to ours. Besides, faster algorithms for those problems under additional assumptions have been studied. For example, Schoot Uiterkamp et al. [41] showed that, if an objective function is a sum of f ( x i + b i ) ( i = 1 , . . . , n ) for some identical convex function f and b i ∈ Z , we can solve Box , Nested , and Laminar with existing algorithms that run in O( n ) [4, 21], O( n log m ) [46], and O( n 2 ) [30] time, respectively. Hochbaum [19] gave an O( n log n log R n ) -time algorithm for Laminar with separable objective functions and no lower bound constraints. 3 Even in those special cases, our results in Table 1 are comparable or better given that prediction errors ∥ x ∗ -ˆ x ∥ 1 are small enough. There also exist empirically fast algorithms [42, 47], whose time complexity bounds are generally worse than the best results. Other problem classes with network and submodular constraints have also been studied [20, 30]. Extending our framework to those classes is left for future work.

Resource allocation with continuous variables has also been well-studied. One may think we can immediately obtain faster algorithms for discrete problems by accelerating continuous optimization algorithms for relaxed problems with predictions and using obtained solutions as warm-starts. However, this is not true since there generally exists an O( n ) gap in the ℓ 1 -norm between real and integer optimal solutions [30, Example 2.9], implying that we cannot always obtain faster algorithms for solving a discrete problem even if an optimal solution to its continuous relaxation is available for free.

2 Preliminaries

Let N := { 1 , . . . , n } be a finite ground set of size n . For i ∈ N , let e i be the i th standard vector, i.e., all zero but the i th entry set to one. For any x ∈ R N and X ⊆ N , let x ( X ) = ∑ i ∈ X x i . Let ⌊·⌉ denote (element-wise) rounding to a closest integer. For a function f : Z N → R ∪ { + ∞} on the integer lattice Z N , its effective domain is defined as dom f := { x ∈ Z N : f ( x ) < + ∞} . A function f is called proper if dom f = ∅ . For Q ⊆ R N , its indicator function δ Q : R N →{ 0 , + ∞} is defined by δ Q ( x ) := 0 if x ∈ Q and + ∞ otherwise.

̸

2.1 M-convexity and greedy algorithm for M-convex function minimization

We briefly explain M-convex functions and sets; see [31, Sections 4 and 6] for details. We say a proper function f : Z N → R ∪{ + ∞} is M-convex if for every x, y ∈ dom f and i ∈ { i ′ ∈ N : x i ′ > y i ′ } , there exists j ∈ { j ′ ∈ N : x j ′ < y j ′ } such that f ( x ) + f ( y ) ≥ f ( x -e i + e j ) + f ( y + e i -e j ) . A non-empty set Q ⊆ Z N is said to be M-convex if its indicator function δ Q : Z N → { 0 , + ∞} is M-convex. Conversely, if f is an M-convex function, dom f is an M-convex set. An M-convex set always lies in a hyperplane defined by { x ∈ R N : x ( N ) = R } for some R ∈ Z . It is worth mentioning that the M-convexity is built upon the well-known basis exchange property of matroids, and the base polytope of a matroid is the convex hull of an M-convex set.

The main subject of this paper is M-convex function minimization, min x ∈ Z N f ( x ) , where f : Z N → R ∪ { + ∞} is an M-convex function. Note that dom f ⊆ Z N represents the feasible region of the problem. For convenience of analysis, we assume the following basic condition.

Assumption 2.1. An M-convex function f : Z N → R ∪ { + ∞} always has a unique minimizer x ∗ .

This uniqueness assumption is common in previous research [39, 10] (and is also needed in [11, 7, 35], although not stated explicitly). In the case of Laminar , it is satisfied for generic, strictly convex objective functions. Even if not, there are natural tie-breaking rules, e.g., choosing the minimizer that attains the lexicographic minimum among all minimizers closest to the origin; we can implement this by adding ϵ ∥ x ∥ 2 2 + ∑ n i =1 ϵ i +1 | x i | for sufficiently small ϵ ∈ (0 , 1) to f , preserving its M-convexity. This is in contrast to the L-convex case, where arbitrarily many minimizers always exist (see [40]).

We can solve M-convex function minimization by a simple greedy algorithm shown in Algorithm 1, which iteratively finds a locally steepest direction, -e i + e j , and proceeds along it. If this update does not improve the objective value, the current solution is ensured to be the minimizer x ∗ = arg min f due to the M-convexity [31, Theorem 6.26]. The number of iterations depends on the ℓ 1 -distance to x ∗ as follows.

Algorithm 1 Greedy algorithm

Proposition 2.2 ([44, Corollary 4.2]) . Algorithm 1 terminates exactly in ∥ x ∗ -x ◦ ∥ 1 / 2 iterations.

A similar iterative method is used in the L-convex case [39], whose number of iterations depends on the ℓ ∞ -distance and a steepest direction is found by some combinatorial optimization algorithm (e.g., the Hopcroft-Karp algorithm in the bipartite-matching case). On the other hand, in the M-convex case, computing a steepest direction is typically cheap (as we only need to find two elements i, j ∈ N ), while the number of iterations depends on the ℓ 1 -distance, which can occupy a larger fraction of the total time complexity than the ℓ ∞ -distance. Hence, reducing the number of iterations with predictions can be more effective in the M-convex case. Section 3 presents a framework for this purpose.

Similar methods to Algorithm 1 are also studied in submodular function maximization [25]. Indeed, M-convex function minimization captures a non-trivial subclass of submodular function maximization that the greedy algorithm can solve (see [31, Note 6.21]), while it is NP-hard in general [32, 13]

3 Warm-start-with-prediction framework M-convex function minimization

We present a framework for warm-starting the greedy algorithm for M-convex function minimization with predictions. Although it resembles those of previous studies [11, 39], it is worth stating explicitly how the time complexity depends on prediction errors for M-convex function minimization.

We consider the following three phases as in previous studies: (i) obtaining an initial feasible solution x ◦ ∈ Z N from a prediction ˆ x ∈ R N , (ii) solving a new instance by warm-starting an algorithm with x ◦ , and (iii) learning predictions ˆ x . The following theorem gives a time complexity bound for (i) and (ii), implying that we can quickly solve a new instance if a given prediction ˆ x is accurate.

Theorem 3.1. Let f : Z N → R ∪ { + ∞} be an M-convex function that has a unique minimizer x ∗ = arg min f and ˆ x ∈ R N be a possibly infeasible prediction. Suppose that Algorithm 1 starts from an initial feasible solution satisfying x ◦ ∈ arg min { ∥ x -⌊ ˆ x ⌉∥ 1 : x ∈ dom f } . Then, Algorithm 1 terminates in O( ∥ x ∗ -ˆ x ∥ 1 ) iterations. Thus, if we can compute x ◦ in T init time and find i, j ∈ N that minimize f ( x -e i + e j ) in Step 3 in T loc time, the total time complexity is O( T init + T loc ∥ x ∗ -ˆ x ∥ 1 ) .

Proof. Due to Proposition 2.2, the number of iterations is bounded by ∥ x ∗ -x ◦ ∥ 1 / 2 . Thus, it suffices to prove ∥ x ∗ -x ◦ ∥ 1 = O( ∥ x ∗ -ˆ x ∥ 1 ) . By using the triangle inequality, we obtain ∥ x ∗ -x ◦ ∥ 1 ≤ ∥ x ∗ -ˆ x ∥ 1 + ∥ ˆ x -⌊ ˆ x ⌉∥ 1 + ∥⌊ ˆ x ⌉-x ◦ ∥ 1 . We below show that the right-hand side is O( ∥ x ∗ -ˆ x ∥ 1 ) . The second term is bounded as ∥ ˆ x -⌊ ˆ x ⌉∥ 1 ≤ ∥ ˆ x -x ∗ ∥ 1 since x ∗ ∈ Z N . As for the third term, we have ∥⌊ ˆ x ⌉-x ◦ ∥ 1 ≤ ∥⌊ ˆ x ⌉-x ∗ ∥ 1 since x ◦ ∈ dom f is a feasible point closest to ⌊ ˆ x ⌉ and x ∗ ∈ dom f , and the right-hand side, ∥⌊ ˆ x ⌉ -x ∗ ∥ 1 , is further bounded as ∥⌊ ˆ x ⌉ -x ∗ ∥ 1 ≤ ∥⌊ ˆ x ⌉ -ˆ x ∥ 1 + ∥ ˆ x -x ∗ ∥ 1 ≤ 2 ∥ ˆ x -x ∗ ∥ 1 due to the previous bound on the second term. Thus, ∥ x ∗ -x ◦ ∥ 1 ≤ 4 ∥ ˆ x -x ∗ ∥ 1 holds.

Note that we round ˆ x to a closest integer point ⌊ ˆ x ⌉ before projecting it onto dom f , while rounding comes after projection in the L-/L ♮ -convex case [39]. This subtle difference is critical since rounding after projection may result in an infeasible integer point that is far from the minimizer x ∗ by O( n ) in the ℓ 1 -norm. To avoid this, we swap the order of the operations and modify the analysis accordingly.

Let us turn to phase (iii), learning predictions. This phase can be done in the same way as previous studies [11, 23]. In particular, we can learn predictions in an online fashion with the standard online subgradient descent method (OSD) as in [23], which works as follows in our case. Let V ⊆ R N be a convex set that we expect to contain an optimal prediction. For any sequence of M-convex functions f 1 , . . . , f T , we regard L t (ˆ x ) := ∥ x ∗ t -ˆ x ∥ 1 for t = 1 , . . . , T as loss functions, where x ∗ t is the minimizer of f t . Fixing ˆ x 1 ∈ V arbitrarily, for t = 1 , . . . , T , OSD iteratively computes a subgradient z t ∈ ∂L t (ˆ x t ) and set ˆ x t +1 = arg min ˆ x ∈ V ∥ ˆ x t -ηz t -ˆ x ∥ 2 , where η > 0 is the step size. OSD returns predictions ˆ x 1 , . . . , ˆ x T that enjoy a regret bound (see, e.g., [33]), and a sample complexity bound follows from online-to-batch conversion [5, 9]. Formally, the following proposition guarantees that we can provably learn predictions to decrease the time complexity bound in Theorem 3.1.

Proposition 3.2 ([23]) . Let f t : Z N → R ∪ { + ∞} for t = 1 , . . . , T be a sequence of M-convex functions, each of which has a unique minimizer x ∗ t = arg min f t , and V ⊆ R N be a closed convex set whose ℓ 2 -diameter is D . Then, OSD with η = D/ √ nT returns ˆ x 1 , . . . , ˆ x T ∈ V that satisfy

Furthermore, for any distribution D over M-convex functions f : Z N → R ∪ { + ∞} , each of which has a unique minimizer x ∗ f = arg min f , δ ∈ (0 , 1] , and ε > 0 , given T = Ω ( ( D ε ) 2 n log 1 δ ) i.i.d. draws, f 1 , . . . , f T , from D , we can compute ˆ x ∈ V that satisfies

with a probability of at least 1 -δ via online-to-batch conversion (i.e., we apply OSD to ∥ x ∗ f t -·∥ 1 for t = 1 , . . . , T and average the outputs).

The convex set V should be designed based on prior knowledge of upcoming instances. For example, previous studies [11, 39] set V = [ -C, + C ] N for some C > 0 , which is expected to contain optimal solutions of all possible instances; then D = 2 C √ n holds. In our case, we sometimes know that the total amount of resources is fixed, i.e., x ( N ) = R , and that every x i is always non-negative. Then, we may set V = { x ∈ [0 , R ] N : x ( N ) = R } , whose ℓ 2 -diameter is D = R √ 2 .

3.1 Time complexity bound for general M-convex function minimization

We here discuss how to apply the above framework to general M-convex function minimization. The reader interested in the main results in Table 1 can skip this section and go to Section 4.

For an M-convex function f : Z N → R ∪ { + ∞} , given access to f 's value and dom f , we can implement the greedy algorithm with warm-starts so that both T init and T loc are polynomially bounded. Suppose that evaluating f ( x ) for any x ∈ Z N takes EO f time. Then, we can find a steepest descent direction at any x ∈ dom f in T loc = O( n 2 EO f ) time by computing f ( x -e i + e j ) for all i, j ∈ N . As for the computation of x ◦ , we need additional information to identify dom f (otherwise, finding any feasible solution may require exponential time in the worst case). Since dom f is an M-convex set, we build on a fundamental fact that any M-convex set can be written as the set of integer points in the base polyhedron of an integer-valued submodular function [15, 31]. A set function ρ : 2 N → R ∪ { + ∞} is called submodular if ρ ( X ) + ρ ( Y ) ≥ ρ ( X ∩ Y ) + ρ ( X ∪ Y ) holds for X,Y ⊆ N , and its base polyhedron is defined as B ( ρ ) := { x ∈ R N : x ( X ) ≤ ρ ( X ) ( X ⊆ N ) , x ( N ) = ρ ( N ) } , where ρ ( ∅ ) = 0 and ρ ( N ) < + ∞ are assumed. Thus, dom f is expressed as dom f = B ( ρ ) ∩ Z N with an integer-valued submodular function ρ : 2 N → Z ∪ { + ∞} . We assume that, for any x ∈ Z N , we can minimize the submodular function ρ + x , defined by ( ρ + x )( X ) := ρ ( X ) + x ( X ) for X ⊆ N , in SFM time. Then, we can obtain x ◦ ∈ dom f from ⌊ ˆ x ⌉ ∈ Z N in T init = O( n SFM ) time, as detailed in Appendix A.1. Therefore, Theorem 3.1 implies the following bound on the total time complexity.

Theorem 3.3. Given a prediction ˆ x ∈ R N , we can minimize f in O( n SFM + n 2 EO f ·∥ x ∗ -ˆ x ∥ 1 ) time.

The current fastest M-convex function minimization algorithms run in O ( n 3 log L n EO f ) and O (( n 3 + n 2 log L n )( log L n / log n ) EO f ) time [43], where L = max { ∥ x -y ∥ ∞ : x, y ∈ dom f } . Thus, our algorithm runs faster if ∥ x ∗ -ˆ x ∥ 1 = o( n ) and SFM = o( n 2 EO f ) . We discuss concrete situations where our approach is particularly effective in Appendix A.2.

4 Laminar convex minimization

This section presents how to obtain the time complexity bounds in Table 1 by applying our framework to laminar convex minimization ( Laminar ) and its subclasses, which are special cases of M-convex function minimization (see [31, Section 6.3]). We first introduce the problem setting of Laminar .

A laminar F ⊆ 2 N is a set family such that for any X,Y ∈ F , either X ⊆ Y , X ⊇ Y , or X ∩ Y = ∅ holds. For convenience, we suppose that F satisfies the following basic properties without loss of generality: ∅ ∈ F , N ∈ F , and { i } ∈ F for every i ∈ N . Then, Laminar is formulated as follows:

where each f Y : R → R ( Y ∈ F ) is a univariate convex function that can be evaluated in O(1) time, R ∈ Z , and ℓ Y ∈ Z ∪{-∞} and u Y ∈ Z ∪{ + ∞} for Y ∈ F . We denote the objective function by f sum ( x ) := ∑ Y ∈F f Y ( x ( Y )) . We let f : Z N → R ∪{ + ∞} be a function such that f ( x ) = f sum ( x ) if x satisfies the constraints in (2) and f ( x ) = + ∞ otherwise; then, f is M-convex and dom f ⊆ Z N represents the feasible region of (2). Nested is a special case where f sum is written as ∑ i ∈ N f i ( x i ) and { Y ∈ F : | Y | ≥ 2 } = { Y 1 , Y 2 , . . . , Y m } consists of nested subsets, i.e., Y 1 ⊂ Y 2 ⊂ · · · ⊂ Y m , and Box is a special case of Nested without nested-subset constraints. Note that our framework in Section 3 only requires the ground set N to be identical over instances. Therefore, we can use it even when both objective functions and constraints change over instances.

It is well known that we can represent a laminar F ⊆ 2 N by a tree T F = ( V , E ) . The vertex set is V = F \{∅} . For Y ∈ V \ { N } , we call X ∈ V a parent of Y if X is the unique minimal set that properly contains Y ; let p ( Y ) ∈ V denote the parent of Y . We call Y ∈ V \ { N } a child of X if p ( Y ) = X . This parent-child relation defines the set of edges as E = { ( X,Y ) : X,Y ∈ V , p ( Y ) = X } . Note that each { i } ∈ V corresponds to a leaf and that N ∈ V is the root. For simplicity, we below suppose the tree T F = ( V , E ) to be binary without loss of generality. If a parent has more than two children, we can recursively divide them into one and the others, which only doubles the number of vertices. Figure 1 illustrates a tree T F of a laminar F = {∅ , { 1 } , { 2 } , { 3 } , { 1 , 2 } , { 1 , 2 , 3 }} .

Applications of Laminar include resource allocation [30], equilibrium analysis of network congestion games [16], and inventory and portfolio management [8]. We below describe a simple example so that the reader can better grasp the image of Laminar ; we will also use it in the experiments in Section 5.

Example: staff assignment. We consider assigning R staff members to n tasks, which form the ground set N . Each task is associated with a higher-level task. For example, if staff members have completed tasks 1 , 2 ∈ N , they are assigned to a new task Y = { 1 , 2 } , which may involve integrating the outputs of the individual tasks. The dependencies among all tasks, including higher-level ones, can be expressed by a laminar F ⊆ 2 N . Each task Y ∈ F is supposed to be done by at least ℓ Y ( ≥ 1 ) and at most u Y ( ≤ R ) members. An employer aims to assign staff members in an attempt to minimize the total perceived workload. For instance, if task i ∈ N requires c i > 0 amount of work and x i staff members are assigned to it, each of them may perceive a workload of f i ( x i ) = c i /x i . Similarly, the perceived workload of task Y = { 1 , 2 } is f Y ( x ( Y )) = c Y /x ( Y ) . The problem of assigning R staff members to n tasks to minimize the total perceived workload, summed over all tasks in F , is formulated

{1}

{2}

{1,2,3}

{3}

𝑥

!

𝑥

"

𝑥

#

{1,2}

I

II

as in (2). Figure 1 illustrates two example assignments, I and II. Here, people assigned to { 1 } and { 2 } must do more tasks than those assigned to { 3 } , and hence assignment I naturally leads to a smaller total perceived workload than II. We can also use any convex function f Y on [ ℓ Y , u Y ] to model other objective functions. Making it faster to solve such problems with predictions enables us to manage massive allocations daily or more frequently.

Our main technical contribution is to obtain the following time complexity bound for Laminar via Theorem 3.1, which also applies to Nested since it is a special case of Laminar .

Theorem 4.1. For Laminar , given a prediction ˆ x ∈ R N , we can obtain an initial feasible solution x ◦ ∈ arg min { ∥ x -⌊ ˆ x ⌉∥ 1 : x ∈ dom f } in T init = O( n ) time and find a steepest descent direction in Step 3 of Algorithm 1 in T loc = O( n ) time. Thus, we can solve Laminar in O( n ∥ x ∗ -ˆ x ∥ 1 ) time.

We prove Theorem 4.1 by describing how to obtain an initial feasible solution and find a steepest descent direction in Sections 4.1 and 4.2, respectively. In Section 4.3, we further reduce the time complexity bound for Box . The algorithmic techniques we use below are not so complicated and can be implemented efficiently, suggesting the practicality of our warm-start-with-prediction framework.

4.1 Obtaining initial feasible solution via fast convex min-sum convolution

We show how to compute x ◦ ∈ dom f in T init = O( n ) time. Given prediction ˆ x ∈ R N , we first compute ⌊ ˆ x ⌉ ∈ Z N in O( n ) time and then solve the following special case of Laminar to obtain x ◦ :

Note that it suffices to find an integer optimal solution to the continuous relaxation of (3) since all the input parameters are integers. Thus, we below discuss how to solve the continuous relaxation of (3).

Solving (3) naively may be as costly as solving the original Laminar instance. Fortunately, however, we can solve it much faster using the special structure of the ℓ 1 -norm objective function. The method we describe below is based on the fast convex min-sum convolution [45], which immediately provides an O( n log 2 n ) -time algorithm for solving (3). We simplify it and eliminate the logarithmic factors by using the fact that the objective function has only two kinds of slopes, ± 1 .

We suppose that each non-leaf vertex Y ∈ V in T F = ( V , E ) has a variable x Y ∈ R , in addition to the original variables x i for leaves { i } ∈ V . We consider assigning a univariate function g : R → R ∪{ + ∞} of the following form to each vertex in V :

-1

+1

𝑏

𝑢

ℓ

𝑔

∗

ℓ

𝑏

𝑢

-1

+1

𝑔

Conjugate

where ℓ, b, u ∈ Z ∪{±∞} and ℓ ≤ b ≤ u . Note that if g is given by (4) up to an additive constant, its convex conjugate g ∗ ( p ) = sup { ⟨ p, x ⟩ -g ( x ) : x ∈ R } is a piecewise-linear function whose slope is ℓ if p ≤ -1 , b if -1 ≤ p ≤ +1 , and u if p ≥ +1 (where ℓ = b and/or b = u can occur). Figure 2 illustrates this conjugate relation. We below construct such functions in a bottom-up manner on T F .

First, we assign function g i ( x i ) = | x i -⌊ ˆ x i ⌉| + δ [ ℓ i ,u i ] ( x i ) to each leaf { i } ∈ V , which represents the i th term of the objective function and the constraint on x i in (3). Next, given two functions g X ( x X ) = | x X -b X | + δ [ ℓ X ,u X ] ( x X ) and g Y ( x Y ) = | x Y -b Y | + δ [ ℓ Y ,u Y ] ( x Y ) of X,Y ∈ V with an identical parent X ∪ Y ∈ V , we construct the parent's function as g X ∪ Y = ( g X □ g Y ) + δ [ ℓ X ∪ Y ,u X ∪ Y ] , where ( g X □ g Y )( x X ∪ Y ) := min { g X ( x X ) + g Y ( x Y ) : x X + x Y = x X ∪ Y } is the infimal convolution. We can confirm that g X ∪ Y also takes the form of (4) as follows. Since g X and g Y are of the form (4), g ∗ X and g ∗ Y have the same breakpoints, ± 1 (see Figure 2). Furthermore, since g X □ g Y = ( g ∗ X + g ∗ Y ) ∗ holds (e.g., [37, Theorem 16.4]), g X □ g Y takes the form of (4) with ℓ = ℓ X + ℓ Y , b = b X + b Y , and u = u X + u Y . Finally, adding δ [ ℓ X ∪ Y ,u X ∪ Y ] preserves the form of (4). We can compute resulting ℓ , b , and u values of g X ∪ Y in O(1) time, and hence we can obtain g Y for all Y ∈ V in a bottom-up manner in O( n ) time. By construction, for each Y ∈ V , g Y ( x Y ) indicates the minimum objective value corresponding to the subtree, ( V Y , E Y ) , rooted at Y when x Y is given. That is, we have

up to constants ignored when constructing g Y , where g Y ( x Y ) = + ∞ if the feasible region is empty. Thus, g N ( R ) corresponds to the minimum value of (3), and our goal is to find integer values x Y for Y ∈ V that attain the minimum value when x N = R ∈ Z is fixed.

Given g Y constructed as above, we can compute desired x Y values in a top-down manner as follows. Let X ∪ Y ∈ V be a non-leaf vertex with two children X and Y . Once x X ∪ Y ∈ dom g X ∪ Y is fixed, we can regard min { g X ( x X ) + g Y ( x Y ) : x X + x Y = x X ∪ Y } ( = g X ∪ Y ( x X ∪ Y ) ) as univariate convex piecewise-linear minimization with variable x X ∈ R (since x Y = x X ∪ Y -x X ), which we can solve in O(1) time. Moreover, since x X ∪ Y and all the parameters of g X and g Y are integers, we can find an integral minimizer x X ∈ Z (and x Y = x X ∪ Y -x X ∈ Z ). Starting from x N = R ∈ Z , we thus compute x Y values for Y ∈ V in a top-down manner, which takes O( n ) time. The resulting x i value for each leaf { i } ∈ V gives the i th element of a desired initial feasible solution x ◦ ∈ dom f .

4.2 Finding steepest descent direction via dynamic programming

Wepresent a dynamic programming (DP) algorithm to find a steepest descent direction in T loc = O( n ) time. Our algorithm is an extension of that used in [30]. The original algorithm finds i that minimizes f ( x -e i + e j ) for a fixed j in O( n ) time. We below extend it to find a pair of ( i, j ) in O( n ) time.

Let x ∈ dom f be a current solution before executing Step 3 in Algorithm 1. We define a directed edge set, A x , on the vertex set V as follows:

Note that x -e i + e j is feasible if and only if ( V , A x ) has a directed path from { i } ∈ V to { j } ∈ V . We then assign an edge weight w X,Y to each ( X,Y ) ∈ A x defined as

By the convexity of f Y , we have w p ( Y ) ,Y ≥ w Y,p ( Y ) , i.e., there is no negative cycle. If x -e i + e j is feasible, f sum ( x -e i + e j ) -f sum ( x ) is equal to the length of a shortest path from { i } to { j } with respect to the edge weights w X,Y (see [30, Section 3.3]). Therefore, finding a steepest descent direction, -e i + e j , reduces to the problem of finding a shortest leaf-to-leaf path in this (bidirectional) tree T x := ( V , A x ) . Constructing this tree takes O( n ) time.

We present a DP algorithm for finding a shortest leaf-to-leaf path. For Y ∈ V , we denote by T x ( Y ) the subtree of T x rooted at Y . Let L Y ↑ be the length of a shortest path from a leaf to Y in T x ( Y ) , L Y ↓ the length of a shortest path from Y to a leaf in T x ( Y ) , and L Y △ the length of a shortest path between any leaves in T x ( Y ) . Clearly, L Y ↑ = L Y ↓ = L Y △ = 0 holds if Y is a leaf in T x . For a non-leaf vertex Y ∈ V , let C ( Y ) denote the set of children of Y in T x . We have the following recursive formulas:

where we regard the minimum on an empty set as + ∞ . Note that, if shortest leaf-toY and Y -to-leaf paths in T x ( Y ) are not edge-disjoint, there must be a leaf-to-leaf simple path in T x ( Y ) whose length is no more than L Y ↑ + L Y ↓ since no negative cycle exists. According to these recursive formulas, we can compute L Y ↑ , L Y ↓ , and L Y △ for all Y ∈ V in O( n ) time by the bottom-up DP on T x . Then, L N △ is the length of a desired shortest leaf-to-leaf path, and its leaves { i } , { j } ∈ V can be obtained by backtracking the DP table in O( n ) time. Thus, we can find a desired direction -e i + e j in O( n ) time.

4.3 Faster steepest descent direction finding for box-constrained case

We focus on Box (1) and present a faster method to find a steepest descent direction, which takes only T loc = O(log n ) time after an O( n ) -time pre-processing. Note that we can obtain an initial feasible solution with the same method as in Section 4.1; hence T init = O( n ) also holds in the Box case.

Theorem 4.2. For Box , given a prediction ˆ x ∈ R N , after an O( n ) -time pre-processing (that can be included in T init = O( n ) ), we can find a steepest descent direction in Step 3 of Algorithm 1 in T loc = O(log n ) time. Thus, we can solve Box in O( n +log n · ∥ x ∗ -ˆ x ∥ 1 ) time.

Proof. In the Box case, f ( x -e i + e j ) -f ( x ) = f i ( x i -1) -f i ( x i )+ f j ( x j +1) -f j ( x j ) holds if x and x -e i + e j are feasible. Furthermore, we only need to care about the box constraints, ℓ i ≤ x i ≤ u i for i = 1 , . . . , n (since x ( N ) = R is always satisfied due to the update rule). Therefore, by keeping ∆ -k := f k ( x k -1) -f k ( x k )+ δ [ ℓ k +1 ,u k ] ( x k ) and ∆ + k := f k ( x k +1) -f k ( x k )+ δ [ ℓ k ,u k -1] ( x k ) values for k = 1 , . . . , n with two min-heaps, respectively, we can efficiently find i ∈ arg min { ∆ -k } n k =1 and j ∈ arg min { ∆ + k } n k =1 ; then, -e i + e j is a steepest descent direction. More precisely, at the beginning of Algorithm 1, we construct the two heaps that maintain ∆ -k and ∆ + k values, respectively, and two arrays that keep track of the location of each element in the heaps; this pre-processing takes O( n ) time. Then, in each iteration of Algorithm 1, we can find a steepest descent direction -e i + e j , update ∆ -i , ∆ + i , ∆ -j , and ∆ + j values (by the so-called increase-/decrease-key operations), and update the heaps and arrays in T loc = O(log n ) time. Thus, Theorem 3.1 implies the time complexity.

0

50

100

t

1000

2000

Iterations

σ

= 1

Learn

Relax

Cold

0

50

100

t

σ

= 5

Learn

Relax

Cold

0

50

100

t

σ

= 10

Learn

Relax

Cold

0

50

100

t

σ

= 20

Learn

Relax

Cold

5 Experiments

We complement our theoretical results with experiments. We used MacBook Air with Apple M2 chip, 24 GB of memory, and macOS Ventura 13.2.1. We implemented algorithms in Python 3.9.12 with libraries such as NumPy 1.23.2. We used Gurobi 10.0.1 [17] for a baseline method explained later. The source code is available at https://github.com/ssakaue/alps-m-convex-code .

5.1 Staff assignment

We consider Laminar instances of the staff-assignment setting described in Section 4. Suppose that we have R = 12800 staff members and n = 128 tasks. Let T F = ( V , E ) be a complete binary tree with n leaves. Define an objective function and inequality constraints as f sum ( x ) = ∑ Y ∈V c Y /x ( Y ) and ℓ Y ≤ x ( Y ) ≤ R for Y ∈ F \ {∅ , N } , respectively, with c Y and ℓ Y values defined as follows. We set c Y = max { ∑ i ∈ Y i + σu a , 1 } , where u a follows the standard normal distribution and σ controls the noise strength. We let ℓ Y = min { 2 h + u b , R/ 2 n -h } , where h ∈ { 0 , 1 , . . . , log n } is the height of Y in T F (a leaf Y has h = 0 ) and u b is drawn uniformly at random from { 0 , 1 , . . . , 50 } ; the minimum with R/ 2 n -h is taken to ensure that the feasible region is non-empty. We thus create a dataset of T = 100 random instances for each σ ∈ { 1 , 5 , 10 , 20 } . We generate 10 such random datasets independently to calculate the mean and standard deviation of the results. The T instances arrive one by one and we learn predictions from optimal solutions to past instances online. By design of c Y , the i th entry of an optimal solution tends to be larger as i increases, which is unknown in advance and should be reflected on predictions ˆ x by learning from optimal solutions to past instances.

We learn predictions ˆ x t ∈ R N for t = 1 , . . . , T by using OSD on V = { x ∈ [0 , R ] N : x ( N ) = R } with a step size of 0 . 01 √ R/n , where the projection onto V is implemented with a technique in [3]. Weuse the all-one vector multiplied by R/n as an initial prediction, ˆ x 0 ∈ V , and set the t th prediction, ˆ x t , to the average of past t outputs, based on online-to-batch conversion. We denote this method by 'Learn.' We also use two baseline methods, 'Cold' and 'Relax', which obtain initial feasible solutions of the greedy algorithm as follows. Cold always uses ˆ x 0 as an initial feasible solution. Relax is a variant of the continuous relaxation approach [30], the fastest method for Laminar with quadratic objectives. Given a new instance, Relax first solves its continuous relaxation (using Gurobi), where the objective function is replaced with its quadratic approximation at ˆ x 0 , and then converts the obtained solution into an initial feasible solution, as with our method. Note that Relax requires information on newly arrived instances, unlike Learn and Cold. Thus, Relax naturally produces good initial feasible solutions while incurring the overhead of solving new relaxed problems. We compare those initialization methods in terms of the number of iterations of the greedy algorithm.

Figure 3 compares Learn, Relax, and Cold for each noise strength σ . Learn always outperforms Cold, and it does even Relax if σ ≤ 10 , suggesting that under moderate noise levels, learning predictions from past optimal solutions can accelerate the greedy algorithm more effectively than solving the relaxed problem of a new instance. The advantage of Learn decreases as σ increases, as expected.

5.2 Resource allocation

We also present experiments using Nested instances of [47, Section 6.3], which include three types of problems, denoted by F , CRASH , and FUEL . The objective functions of F , CRASH , and FUEL are written as ∑ n i =1 f i ( x i ) where f i ( x i ) = x 4 i / 4 + p i x i , f i ( x i ) = k i + p i /x i , and f i ( x i ) = p i · c i ( c i /x i ) 3 , respectively, with some input parameters p i , k i , c i . F is a synthetic problem of optimizing the

0

50

100

t

0

200

Iterations

σ

= 0

.

1

Learn

Relax

Cold

0

50

100

t

σ

= 1

Learn

Relax

Cold

0

50

100

t

σ

= 10

Learn

Relax

Cold

(a)

F

0

50

100

t

200

400

Iterations

σ

= 0

.

1

Learn

Relax

Cold

0

50

100

t

σ

= 1

Learn

Relax

Cold

0

50

100

t

σ

= 10

Learn

Relax

Cold

(b)

CRASH

0

50

100

t

0

25

50

Iterations

σ

= 0

.

1

Learn

Relax

Cold

0

50

100

t

σ

= 1

Learn

Relax

Cold

0

50

100

t

σ

= 10

Learn

Relax

Cold

(c)

FUEL

fourth-order polynomial, while CRASH and FUEL come from real-world project crashing and ship speed optimization, as noted in [46]. We create Nested instances with n = 100 for F , CRASH , and FUEL based on the procedure of [47]. We then generate T = 100 instances by perturbing parameters defining constraints and objectives with Gaussian noises multiplied by σ = 0 . 1 , 1 . 0 , 10 . 0 , which controls the noise strength. As with the previous experiments, we measure the number of iterations of the greedy algorithm initialized by Cold, Relax, and Learn over the 100 instances. Regarding Learn, we set OSD's step size in the same manner as the previous experiments, and each element of an initial prediction ˆ x 0 is set to ⌊ R/n ⌋ or ⌊ R/n ⌋ +1 at random so that ˆ x 0 ( N ) = R holds.

Figure 4 shows the results. Similar to the previous synthetic setting, Learn attains fewer iterations than Relax and Cold for CRASH and FUEL with moderate noise strengths ( σ = 0 . 1 , 1 . 0 ). As for F , Relax performs extremely well and surpasses Learn, probably because the synthetic fourthorder polynomial objective is easy to handle with the continuous-relaxation method used in Relax. Nevertheless, it is significant that Learn can surpass Relax for CRASH and FUEL , which come from real-world applications, under moderate noises.

6 Conclusion and limitations

We have extended the idea of warm-starts with predictions to M-convex function minimization. By combining our framework with algorithmic techniques, we have obtained specific time complexity bounds for Laminar , Nested , and Box . Those bounds can be better than the current best worst-case bounds given accurate predictions, which we can provably learn from past data. Experiments have confirmed that using predictions reduces the number of iterations of the greedy algorithm.

Since our focus is on improving theoretical guarantees with predictions, further study of practical aspects is left for future work. While we have used the standard OSD for learning predictions, we expect that more sophisticated learning methods can further improve the empirical performance. Also, extending the framework to other problem classes is an exciting future direction. A technical open problem is eliminating Assumption 2.1, although it hardly matters in practice. We expect the idea of [40] for the L-/L ♮ -convex case is helpful, but it seems more complicated in the M-convex case.

Acknowledgments and Disclosure of Funding

The authors thank Satoru Iwata for telling us about the integrality of the dual problem of submodular function minimization. The authors also thank the anonymous reviewers for their helpful comments. This work was supported by JST ERATO Grant Number JPMJER1903 and JSPS KAKENHI Grant Number JP22K17853.

References

Y. Azar, D. Panigrahi, and N. Touitou. Online graph algorithms with predictions. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2022) , pages 35-66. SIAM, 2022 (cited on page 2).

E. Bamas, A. Maggiori, and O. Svensson. The primal-dual method for learning augmented algorithms. In Advances in Neural Information Processing Systems (NeurIPS 2020) , volume 33, pages 20083-20094. Curran Associates, Inc., 2020 (cited on page 2).

C. Bauckhage. NumPy/SciPy recipes for data science: projections onto the standard simplex. Technical report, 2020 (cited on page 9).

P. Brucker. An O ( n ) algorithm for quadratic knapsack problems. Operations Research Letters , 3(3):163-166, 1984 (cited on page 3).

N. Cesa-Bianchi, A. Conconi, and C. Gentile. On the generalization ability of on-line learning algorithms. IEEE Transactions on Information Theory , 50(9):2050-2057, 2004 (cited on page 5).

D. Chakrabarty, P. Jain, and P. Kothari. Provable submodular minimization using Wolfe's algorithm. In Advances in Neural Information Processing Systems (NuerIPS 2014) , volume 27, pages 802-809. Curran Associates, Inc., 2014 (cited on page 15).

J. Chen, S. Silwal, A. Vakilian, and F. Zhang. Faster fundamental graph algorithms via learned predictions. In Proceedings of the 39th International Conference on Machine Learning (ICML 2022) , volume 162, pages 3583-3602. PMLR, 2022 (cited on pages 1-4).

X. Chen and M. Li. M ♮ -convexity and its applications in operations. Operations Research , 69(5):1396-1408, 2021 (cited on pages 2, 6).

A. Cutkosky. Anytime online-to-batch, optimism and acceleration. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019) , volume 97, pages 1446-1454. PMLR, 2019 (cited on page 5).

S. Davies, B. Moseley, S. Vassilvitskii, and Y . Wang. Predictive flows for faster Ford-Fulkerson. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023) , volume 202, pages 7231-7248. PMLR, 2023 (cited on pages 1, 3, 4).

M. Dinitz, S. Im, T. Lavastida, B. Moseley, and S. Vassilvitskii. Faster matchings via learned duals. In Advances in Neural Information Processing Systems (NeurIPS 2021) , volume 34, pages 10393-10406. Curran Associates, Inc., 2021 (cited on pages 1-5).

J. Edmonds. Submodular functions, matroids, and certain polyhedra. In Combinatorial Structures and Their Applications , pages 69-87. Gordon and Breach, New York, 1970 (cited on page 14).

U. Feige. A threshold of ln n for approximating set cover. Journal of the ACM , 45(4):634-652, 1998 (cited on page 4).

G. N. Frederickson and D. B. Johnson. The complexity of selection and ranking in X + Y and matrices with sorted columns. Journal of Computer and System Sciences , 24(2):197-208, 1982 (cited on page 2).

S. Fujishige. Submodular Functions and Optimization , volume 58 of Annals of Discrete Mathematics . Elsevier, Amsterdam, second edition, 2005 (cited on pages 5, 14).

S. Fujishige, M. Goemans, T. Harks, B. Peis, and R. Zenklusen. Congestion games viewed from M-convexity. Operations Research Letters , 43(3):329-333, 2015 (cited on pages 2, 6).

Gurobi Optimization, LLC. Gurobi optimizer reference manual. https://www.gurobi.com , 2023 (cited on page 9).

D. S. Hochbaum and J. G. Shanthikumar. Convex separable optimization is not much harder than linear optimization. Journal of the ACM , 37(4):843-862, 1990 (cited on page 2).

D. S. Hochbaum. Lower and upper bounds for the allocation problem and other nonlinear optimization problems. Mathematics of Operations Research , 19(2):390-409, 1994 (cited on pages 2, 3).

D. S. Hochbaum and S.-P. Hong. About strongly polynomial time algorithms for quadratic optimization over submodular constraints. Mathematical Programming , 69(1-3):269-309, 1995 (cited on pages 3, 15).

T. Ibaraki and N. Katoh. Resource Allocation Problems: Algorithmic Approaches . MIT Press, first edition, 1988 (cited on page 3).

N. Katoh, A. Shioura, and T. Ibaraki. Resource allocation problems. In Handbook of Combinatorial Optimization , pages 2897-2988. Springer, New York, NY, 2013 (cited on pages 2, 3).

M. Khodak, M.-F. Balcan, A. Talwalkar, and S. Vassilvitskii. Learning predictions for algorithms with predictions. In Advances in Neural Information Processing Systems (NeurIPS 2022) , volume 35, pages 3542-3555. Curran Associates, Inc., 2022 (cited on pages 2, 5).

S. Lacoste-Julien and M. Jaggi. On the global linear convergence of Frank-Wolfe optimization variants. In Advances in Neural Information Processing Systems (NeurIPS 2015) , volume 28, pages 496-504. Curran Associates, Inc., 2015 (cited on page 15).

J. Lee, V . S. Mirrokni, V . Nagarajan, and M. Sviridenko. Maximizing nonmonotone submodular functions under matroid or knapsack constraints. SIAM Journal on Discrete Mathematics , 23(4):2053-2078, 2010 (cited on page 4).

Y. T. Lee, A. Sidford, and S. C.-W. Wong. A faster cutting plane method and its implications for combinatorial and convex optimization. In Proceedings of the 56th Annual Symposium on Foundations of Computer Science (FOCS 2015) , pages 1049-1065, 2015 (cited on page 15).

A. Lindermayr and N. Megow. https://algorithms-with-predictions.github.io/ (cited on page 3).

T. Lykouris and S. Vassilvitskii. Competitive caching with machine learned advice. Journal of the ACM , 68(4):1-25, 2021 (cited on page 2).

M. Mitzenmacher and S. Vassilvitskii. Algorithms with predictions. In Beyond the Worst-Case Analysis of Algorithms , pages 646-662. Cambridge University Press, Cambridge, 2021 (cited on pages 1, 2).

S. Moriguchi, A. Shioura, and N. Tsuchimura. M-convex function minimization by continuous relaxation approach: proximity theorem and algorithm. SIAM Journal on Optimization , 21(3):633-668, 2011 (cited on pages 3, 6, 8, 9, 15).

K. Murota. Discrete Convex Analysis , volume 10 of Monographs on Discrete Mathematics and Applications . SIAM, Philadelphia, 2003 (cited on pages 1, 3-6, 14).

G. L. Nemhauser and L. A. Wolsey. Best algorithms for approximating the maximum of a submodular set function. Mathematics of Operations Research , 3(3):177-188, 1978 (cited on page 4).

F. Orabona. A Modern Introduction to Online Learning . OpenBU, 2020 (cited on page 5).

J. B. Orlin and B. Vaidyanathan. Fast algorithms for convex cost flow problems on circles, lines, and trees. Networks , 62(4):288-296, 2013 (cited on page 2).

A. Polak and M. Zub. Learning-augmented maximum flow. arXiv:2207.12911 , 2022 (cited on pages 1, 3, 4).

M. Purohit, Z. Svitkina, and R. Kumar. Improving online algorithms via ML predictions. In Advances in Neural Information Processing Systems (NeurIPS 2018) , volume 31, pages 96849693. Curran Associates, Inc., 2018 (cited on page 2).

R. T. Rockafellar. Convex Analysis , volume 13 of Princeton Landmarks in Mathematics and Physics . Princeton University Press, Princeton, 1970 (cited on page 7).

T. Roughgarden. Beyond the Worst-Case Analysis of Algorithms . Cambridge University Press, Cambridge, 2021 (cited on page 2).

S. Sakaue and T. Oki. Discrete-convex-analysis-based framework for warm-starting algorithms with predictions. In Advances in Neural Information Processing Systems (NeurIPS 2022) , volume 35, pages 20988-21000. Curran Associates, Inc., 2022 (cited on pages 1-5).

S. Sakaue and T. Oki. Rethinking warm-starts with predictions: learning predictions close to sets of optimal solutions for faster L-/L ♮ -convex function minimization. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023) , volume 202, pages 2976029776. PMLR, 2023 (cited on pages 4, 10).

M. H. H. Schoot Uiterkamp, M. E. T. Gerards, and J. L. Hurink. On a reduction for a class of resource allocation problems. INFORMS Journal on Computing , 34(3):1387-1402, 2022 (cited on pages 3, 15).

M. H. H. Schoot Uiterkamp, J. L. Hurink, and M. E. T. Gerards. A fast algorithm for quadratic resource allocation problems with nested constraints. Computers & Operations Research , 135:105451, 2021 (cited on page 3).

A. Shioura. Fast scaling algorithms for M-convex function minimization with application to the resource allocation problem. Discrete Applied Mathematics , 134(1):303-316, 2004 (cited on pages 2, 5, 14).

A. Shioura. M-convex function minimization under L1-distance constraint and its application to dock reallocation in bike-sharing system. Mathematics of Operations Research , 47(2):15661611, 2022 (cited on page 4).

P. Tseng and Z.-Q. Luo. On computing the nested sums and infimal convolutions of convex piecewise-linear functions. Journal of Algorithms & Computational Technology , 21(2):240266, 1996 (cited on page 7).

T. Vidal, D. Gribel, and P. Jaillet. Separable convex optimization with nested lower and upper constraints. INFORMS Journal on Optimization , 1(1):71-90, 2019 (cited on pages 2, 3, 10).

Z. Wu, K. Nip, and Q. He. A new combinatorial algorithm for separable convex resource allocation with nested bound constraints. INFORMS Journal on Computing , 33(3):1197-1212, 2021 (cited on pages 3, 9, 10).

Peer reviews:
comments: SUMMARY:
In the vein of the seminal work by Sakaue and Oki for L-convex functions, this paper proposes a new method to accelerate M-convex function minimization using past predictions, a technique known as warm-start. 

The contributions are as follows:
1) Present a framework to accelerate M-convex function minimization with past predictions
2) Applying this framework to the Laminar, Box and Nested classes of problems and improving time bounds using warm start for each of these problems

STRENGTHS:
The results in the experiments section seems promising. Although I'm not an expert in discrete optimization, the problems tackled seemed of great importance.

WEAKNESSES:
1) This paper is very theoretically dense and tries to tackle a lot of problems at once, the contributions are not clear. I think this paper would gain at focusing solely on the Laminar problem and possibly present the extension of your work in appendix. The demonstration would be clearer, and your paper more straight to the point.

2) Although rigorously written, it feels like this paper is a re-adaptation of the work from Sakaue and Oki to the class of M-convex functions and is thus not particularly original. 

3) The results section is rather light compared to the extent of the theoretical claims on the Laminar, Box and Nested subproblems. I'd have preferred for each of the 3 classes of problems you mention, a first experiment on synthetic data with comparison between warm start and no warm-start. Then, a second experiment on real-world data to prove the superiority of your method.

QUESTIONS:
- In figure 3, the y-axis label is set to "Iterations". What is the meaning of these iterations? Shouldn't this be the suboptimality gap?

LIMITATIONS:
The limitations are addressed.
No negative societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper applies the learning-augmented algorithms framework to a class of discrete optimization problems called M-convex. A function defined on an integer grid is M-convex if for every x, y it holds that f(x)+f(y) >= f(x-ei+ej)+f(y+ei-ej) for some base vectors ei, ej. The paper complements the line of research on improving running time of static algorithms using predictions, started by Dinitz et al. (NeurIPS’21), and is a natural followup to the Sakaue&Oki’s paper (NeurIPS’22) on another class, called L-convex.\n\nThe authors propose a general framework for solving M-convex optimization problems given a predicted optimal solution that is supposed to be close to the true one. Then, they give algorithms for specific subclasses of these problems, called Laminar, Nested, and Box (after the form of constraints allowed in each class). The running times they obtain are O(n*eta) for Laminar and Nested and O(n+log(n)*eta) for Box, where eta = ||x_opt - x_pred||_1 is the L1-error of the prediction.\n\nSTRENGTHS:\nThere are nice algorithms in Sections 4.1 and 4.2 (for rounding to the nearest feasible solution and for finding the steepest descent in the laminar case).\n\nWith (very) accurate predictions, the algorithm for the special case of box constraints can go (a tiny bit) below a known lower bound for classic (prediction-less) algorithms, and it seems to be the first such example in the literature about learning-augmented static algorithms.\n\nThe paper is written with care and not hard to follow.\n\nWEAKNESSES:\nThe algorithm/framework for general M-convex optimization is technically trivial. The algorithms for the laminar subclass are much more interesting, but I’m not convinced that this subclass is interesting enough on its own for the results to have substantial implications.\n\nInteresting technical ideas seem very much tailored to the specific setup, so it might be difficult for this paper to have a larger impact or inspire further research.\n\nThe experiments are very simplistic – on a toy problem with synthetic data and predictions. (Still it was somewhat surprising to learn that, for a concentrated enough distribution of instances, an integral solution learned from previous instances can be a better initialization than an optimal solution to the fractional relaxation.) If I consider this paper to be a pure theory paper (which for now I do), then such experiments are not an issue, but in that case the theoretical contributions of the paper might not be strong enough to pass the ICML bar.\n\nQUESTIONS:\nCould you find some real-world applications of laminar convex optimization that have existing benchmarks/datasets, and run experiments on them? (Even with some synthetic method for generating predictions.)\n\nStatement in lines 59–60 seems not accurate. For instance, the negative-weight SSSP algorithm of [7] runs in O(m) time for very good predictions, beating the best known worst-case bound by a couple of log factors.\n\nI do not agree with the statement in line 390 that you can beat best known algorithms “(...) given accurate predictions, which we can provably learn from past data.” The learnability proofs tell you that you can efficiently generate predictions as good as possible in a given context, but not that there exist predictions with small enough error. (This is a common limitation of PAC-learning framework, so I’m not complaining about your result, just about the way in which you describe it.)\n\nMinor comments:\n\nLine 26: At first it was totally unclear to me what “conjugate” was supposed to mean in this context.\n\nLine 28: “Specifically, we focus on an important subclass (...)” – this sentence is misleading as it suggests that you only focus on the subclass, which is not the case, because you address the general problem as well.\n\nLine 30: “(...) is widely studied (...)” – could you please provide some references?\n\nLine 42: Please mention that the greedy algorithm is given later as Algorithm 1.\n\nLine 101: “closet” -> “closest”\n\nLine 223: For readability, I’d consider using a colon instead of \\mid because \\mid blends with the set cardinality symbol.\n\nLIMITATIONS:\nTreating it as a theory paper, it does not have any important limitations (apart from the possibly limited impact itself). The title might be slightly misleading, since the most interesting results are only for a relatively narrow special case (laminar) and not the general case mentioned in the title (L-convex).", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe goal of the work is to minimize M-convex functions with prediction. The work mainly focuses on a subclass of M-convex functions that use Laminar, Nested or Box constraints.\n\nSTRENGTHS:\nMinimizing M-convex functions is an important class of discrete optimization problems that has wide variety of applications. Using algorithms with prediction is an important area of research that can typically solve many problems in discrete convex analysis. Especially due to wide variety of application.\n\nSoundness. The work is technically sound that uses relevant algorithms produced in literature to solve a very important problem.\n\nPresentation. The paper is very well written that flows flawlessly with very minor concerns.\n\nContribution. The contribution is first of its kind to the class of problems that it is applied it. However, it must be noted that most of the major algorithms used in the paper are proposed in literature. The main contribution is modification of these classical algorithms to solve the bigger problem of minimizing M-convex functions with predictions.\n\nWEAKNESSES:\nI dont find many weaknesses in this work.\n\nQUESTIONS:\nI have two major concerns with the work\n\n1. In section 4.2, it is clear that finding the node $j$ for a fixed node $i$ can be done in $O(n)$ time. However, I think it is not  $O(n)$ as the Dynamic programming algorithm iterates over $\\mathcal{C}(Y)$ and this is not a constant which depends on the type of laminar function. On the other hand, such a tree is much simpler for a nested case or box case.\n\n2. For the sake of clarity, I request the authors to also explain the learning predictions part even though it is the same as used in other seminal works at least in the appendix. The paper is not self contained otherwise.\n\nLIMITATIONS:\nThis work would have been more complete if more efficient algorithms could be proposed for submodular constraints. This is the only reason for me to not give strong accept. Although laminar, nested and box are interesting, I believe, they have limited application scope.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nExtending the warm-starting techniques in L-convex function minimization by Sakaue and Oki (2022), the authors study the problem of acclerating M-convex function minimization with predictions. The idea is to start from a (possibly infeasible) predicted solution, project the rounded solution to the feasible region and then apply the standard M-convex function minimization greedy algorithm. In particular, the authors show that when applied to Laminar convex minimization, a special case of M-convex function minimization, their framework can achieve better time complexity than current worst-case time complexity provided that the prediction is accurate enough. Experiments on the staff-assignment problem confirm that the proposed framework can help reduce the number of iterations of the greedy algorithm.\n\nSTRENGTHS:\nThe paper studies an interesting problem, and is written well in terms of structure, clarity in explanation and technical presentation. Theoretical results are justified by empirical experiments.\n\nWEAKNESSES:\n1. The framework proposed in this paper for M-convex optimization is somewhat similar to the one in Sakaue and Oki (2022) for L-convex optimization. So novelty in the general framework is limited.\n2. The authors provide comparision between their results and worst-case time complexity for laminar convex minimization. However, it seems like their complexity outperforms the worst-case only under the assumption of highly accurate predictions (small enough $\\ell_1$ norm prediction error), which does not necessarily hold in practice.\n\nQUESTIONS:\nThe staff assignment problem does not seem to be a very realistic test problem to me (with 12800 staff members and 128 tasks). Are there more standard test problems? How does the proposed framework perform on them?\n\nLIMITATIONS:\nThe limitations have been addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies some classes of $M$-convex minimization problems, to which the recent framework of \"warm-starts with predictions\" is applied. The paper provides provable time complexity bounds on the standard greedy algorithm for $M$-convex minimization where the bounds are dependent on the $\\ell_1$-distance between the optimal solution and the predicted initial solution. The theoretical performance guarantees are promising and improve upon the existing methods that are not using predictions. At the same time, however, one would argue that the technical contributions of this paper are limited in that the results rely on and are deduced by applications of the existing results in the literature. Furthermore, one would be interested to see the numerical impact of the framework on more concrete problem settings, e.g., portfolio management and resource allocation, which were not tested in this paper.\n\nSTRENGTHS:\n* The time complexity bounds provided in this paper are the first results that analyze the performance of the framework of warm-starts with predictions applied to $M$-convex function minimization.\n\n* The time complexity bounds improve upon the existing bounds for some classes of $M$-convex minimization problems when we may obtain an accurate prediction where the accuracy is measured by the $\\ell_1$-distance to the (unique) optimal solution.\n\nWEAKNESSES:\n* ​The technical contributions of this paper are limited. The greedy algorithm and main results of this paper are built upon and follow from [(Shioura (2022), Corollary 4.2)](https://pubsonline.informs.org/doi/abs/10.1287/moor.2021.1180) which gives an upper bound on the number of required iterations for the greedy algorithm in terms of the proximity term. The rest of the results are basically about bounding $T_{\\text{init}}$, the time required to convert a prediction to a feasible solution, and $T_{\\text{loc}}$, the time bound for computing a locally steepest direction. Even these results follow from standard techniques in the literature.\n\n* This paper lacks computational demonstration. $M$-convex minimization has applications in resource allocation, equilibrium analysis, and portfolio management, but none of these problems were tested. In particular, one would be interested in how well the framework of this paper performs for the operations management models studied in [(Chen and Li, (2021))](https://pubsonline.informs.org/doi/abs/10.1287/opre.2020.2070). Furthermore, the numerical results reported in Section 6 do not consider the methods against which the theoretical complexity bounds are compared.\n\nQUESTIONS:\n* When testing the performance of this paper's framework against the methods without predictions, would it be fair to consider the time to compute a good prediction? Proposition 3.2 provides the regret bound by the standard online learning method, but what would be the required number of iterations to deduce the desired proximity bound on $\\|x^*-\\hat x\\|_1$? Can you compare the total time required by the framework of this paper, adding up the time for learning a good prediction and the time to solve the problem with the warm start, against the methods without predictions?\n\n* You mention that the required framework for $M^\\natural$-convex minimization is similar to that for $M$-convex minimization. Can you provide more details?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper discusses the growing interest in accelerating optimization algorithms using machine-learned predictions. It highlights the work of Sakaue and Oki, who introduced a general framework for employing predictions to warm-start the L-convex function minimization method, demonstrating its effectiveness for various discrete optimization problems. Building on this, the paper presents a new framework that leverages predictions to accelerate M-convex function minimization with improved time complexity bounds, thereby extending the applicability of predictive techniques to a wider range of discrete optimization algorithms.\n\nSTRENGTHS:\nThis paper is well-written, presenting intuitive results. It exhibits technical solidity, providing clear explanations of how the proposed algorithm enhances the existing worst-case time complexity.\n\nWEAKNESSES:\n1. The main theoretical results appear to be straightforward. The computational time complexity relies on the distance between the initialization and the optimal solution, assuming a good initialization is provided. However, one limitation of this paper is the absence of an analysis regarding the methodology for obtaining such a good initialization.\n2. The difficulty in deriving the corresponding theoretical results is not well elucidated in the paper. For instance, proving Theorem 3.1 hinges on Proposition 2.2, which is simply a direct application from existing works [44]. It would be more beneficial if the paper provided a clearer explanation of the technical challenges involved in obtaining the theoretical results.\n3. This paper lacks numerical justifications for the proposed theoretical time complexity bound.\n\nQUESTIONS:\nCould the authors present concrete evidence illustrating how the theoretical bounds offer valuable insights capable of significantly accelerating the practical applications using the proposed algorithm?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-26"}