SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We learn predictions  $\hat{x}_t \in \mathbb{R}^N$  for  $t = 1, \dots, T$  by using OSD on  $V = \{x \in [0, R]^N : x(N) = R\}$  with a step size of  $0.01\sqrt{R / n}$ , where the projection onto  $V$  is implemented with a technique in [3]. We use the all-one vector multiplied by  $R / n$  as an initial prediction,  $\hat{x}_0 \in V$ , and set the  $t$ th prediction,  $\hat{x}_t$ , to the average of past  $t$  outputs, based on online-to-batch conversion. We denote this method by "Learn." We also use two baseline methods, "Cold" and "Relax", which obtain initial feasible solutions of the greedy algorithm as follows. Cold always uses  $\hat{x}_0$  as an initial feasible solution. Relax is a variant of the continuous relaxation approach [30], the fastest method for Laminar with quadratic objectives. Given a new instance, Relax first solves its continuous relaxation (using Gurobi), where the objective function is replaced with its quadratic approximation at  $\hat{x}_0$ , and then converts the obtained solution into an initial feasible solution, as with our method. Note that Relax requires information on newly arrived instances, unlike Learn and Cold. Thus, Relax naturally produces good initial feasible solutions while incurring the overhead of solving new relaxed problems. We compare those initialization methods in terms of the number of iterations of the greedy algorithm.
Figure 3 compares Learn, Relax, and Cold for each noise strength  $\sigma$ . Learn always outperforms Cold, and it does even Relax if  $\sigma \leq 10$ , suggesting that under moderate noise levels, learning predictions from past optimal solutions can accelerate the greedy algorithm more effectively than solving the relaxed problem of a new instance. The advantage of Learn decreases as  $\sigma$  increases, as expected.
5.2 Resource allocation
We also present experiments using Nested instances of [47, Section 6.3], which include three types of problems, denoted by F, CRASH, and FUEL. The objective functions of F, CRASH, and FUEL are written as  $\sum_{i=1}^{n} f_i(x_i)$  where  $f_i(x_i) = x_i^4 / 4 + p_i x_i$ ,  $f_i(x_i) = k_i + p_i / x_i$ , and  $f_i(x_i) = p_i \cdot c_i (c_i / x_i)^3$ , respectively, with some input parameters  $p_i$ ,  $k_i$ ,  $c_i$ . F is a synthetic problem of optimizing the
(a) F
(b) CRASH
(c) FUEL
Figure 4: The number of iterations of the greedy algorithm initialized with Learn, Relax, and Cold. The curve and error band show the mean and standard deviation of 10 independent runs, respectively.
fourth-order polynomial, while CRASH and FUEL come from real-world project crashing and ship speed optimization, as noted in [46]. We create Nested instances with  $n = 100$  for F, CRASH, and FUEL based on the procedure of [47]. We then generate  $T = 100$  instances by perturbing parameters defining constraints and objectives with Gaussian noises multiplied by  $\sigma = 0.1, 1.0, 10.0$ , which controls the noise strength. As with the previous experiments, we measure the number of iterations of the greedy algorithm initialized by Cold, Relax, and Learn over the 100 instances. Regarding Learn, we set OSD's step size in the same manner as the previous experiments, and each element of an initial prediction  $\hat{x}_0$  is set to  $\lfloor R / n \rfloor$  or  $\lfloor R / n \rfloor + 1$  at random so that  $\hat{x}_0(N) = R$  holds.
Figure 4 shows the results. Similar to the previous synthetic setting, Learn attains fewer iterations than Relax and Cold for CRASH and FUEL with moderate noise strengths  $(\sigma = 0.1, 1.0)$ . As for F, Relax performs extremely well and surpasses Learn, probably because the synthetic fourth-order polynomial objective is easy to handle with the continuous-relaxation method used in Relax. Nevertheless, it is significant that Learn can surpass Relax for CRASH and FUEL, which come from real-world applications, under moderate noises.
6 Conclusion and limitations
We have extended the idea of warm-starts with predictions to M-convex function minimization. By combining our framework with algorithmic techniques, we have obtained specific time complexity bounds for Laminar, Nested, and Box. Those bounds can be better than the current best worst-case bounds given accurate predictions, which we can provably learn from past data. Experiments have confirmed that using predictions reduces the number of iterations of the greedy algorithm.
Since our focus is on improving theoretical guarantees with predictions, further study of practical aspects is left for future work. While we have used the standard OSD for learning predictions, we expect that more sophisticated learning methods can further improve the empirical performance. Also, extending the framework to other problem classes is an exciting future direction. A technical open problem is eliminating Assumption 2.1, although it hardly matters in practice. We expect the idea of [40] for the  $\mathrm{L} - / \mathrm{L}^{\natural}$ -convex case is helpful, but it seems more complicated in the M-convex case.
The authors thank Satoru Iwata for telling us about the integrality of the dual problem of submodular function minimization. The authors also thank the anonymous reviewers for their helpful comments. This work was supported by JST ERATO Grant Number JPMJER1903 and JSPS KAKENHI Grant Number JP22K17853.
[40] S. Sakaue and T. Oki. Rethinking warm-starts with predictions: learning predictions close to sets of optimal solutions for faster  $\mathrm{L} - / \mathrm{L}^{\natural}$ -convex function minimization. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), volume 202, pages 29760-29776. PMLR, 2023 (cited on pages 4, 10).
[41] M. H. H. Schoot Uiterkamp, M. E. T. Gerards, and J. L. Hurink. On a reduction for a class of resource allocation problems. INFORMS Journal on Computing, 34(3):1387-1402, 2022 (cited on pages 3, 15).
[42] M. H. H. Schoot Uiterkamp, J. L. Hurink, and M. E. T. Gerards. A fast algorithm for quadratic resource allocation problems with nested constraints. Computers & Operations Research, 135:105451, 2021 (cited on page 3).
[43] A. Shioura. Fast scaling algorithms for M-convex function minimization with application to the resource allocation problem. Discrete Applied Mathematics, 134(1):303-316, 2004 (cited on pages 2, 5, 14).
[44] A. Shioura. M-convex function minimization under L1-distance constraint and its application to dock reallocation in bike-sharing system. Mathematics of Operations Research, 47(2):1566-1611, 2022 (cited on page 4).
[45] P. Tseng and Z.-Q. Luo. On computing the nested sums and infimal convolutions of convex piecewise-linear functions. Journal of Algorithms & Computational Technology, 21(2):240-266, 1996 (cited on page 7).
[46] T. Vidal, D. Gribel, and P. Jaillet. Separable convex optimization with nested lower and upper constraints. INFORMS Journal on Optimization, 1(1):71-90, 2019 (cited on pages 2, 3, 10).
[47] Z. Wu, K. Nip, and Q. He. A new combinatorial algorithm for separable convex resource allocation with nested bound constraints. INFORMS Journal on Computing, 33(3):1197-1212, 2021 (cited on pages 3, 9, 10).