A Gradient Method for Multilevel Optimization

1 Introduction . Multilevel optimization When modeling real-world problems , there is often a hierarchy of decisionmakers , and decisions are taken at different levels in the hierarchy . In this work , we consider multilevel optimization problems whose simplest form1 is the following : min x1∈S1⊆Rd1 , x∗2 , ... , x∗n f1 ( x1 , x ∗ 2 , . . . , x ∗ n ) s.t . x∗2 = argmin x2∈Rd2 , x∗3 , ... , x∗n f2 ( x1 , x2 , x ∗ 3 , . . . , x ∗ n ) s.t. . . . x∗n = argmin xn∈Rdn fn ( x1 , x2 , . . . , xn ) , ( 1 ) where di is positive integers , xi ∈ Rdi is the decision variable at the ith level , and fi : Rdi → R is the objective function of the ith level for i = 1 , . . . , n. In the formulation , an optimization problem contains another optimization problem as a constraint . The framework can be used to formulate problems in which decisions are made in sequence and earlier decisions influence later decisions . An optimization problem in which this structure is n-fold is called an n-level optimization problem or a multilevel optimization problem especially when n ≥ 3 . Existing study on bilevel optimization Especially , when n = 2 in ( 1 ) , it is called a bilevel optimization problem . Bilevel optimization or multilevel optimization has been a well-known problem in the field of optimization since the 1980s or 1990s respectively ( see the survey paper [ 24 ] for the research at that time ) , and their solution methods have been studied mainly on bilevel 1This assumes that lower-level problems have unique optimal solutions x∗2 , . . . , x∗n for the simplicity . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . optimization until now . Recently , studies on bilevel optimization have received significant attention from the machine learning community due to practical applications and the development of efficient algorithms . For example , bilevel optimization was used to formulate decision-making problems for players in conflict [ 25 , 22 ] , and to formulate hyperparameter learning problems in machine learning [ 5 , 11 , 12 ] . Franceschi et al . [ 11 ] constructed a single-level problem that approximates a bilevel problem by replacing the lower-level problem with T steepest descent update equations using a prechosen iteration number T . It is shown that the optimization problem with T steepest descent update equations asymptotically converges to the original bilevel problem as T → ∞ . It is quite different from the ordinary approach ( e.g. , [ 1 ] ) that transforms bilevel optimization problems into single-level problems using the optimality condition of the lower-level problem . The application of the steepest descent method to bilevel optimization problems has recently attracted attention , resulting in a stream of papers ; e.g. , [ 13 , 20 , 18 ] . Existing study on multilevel optimization Little research has been done on decision-making models ( 1 ) that assume the multilevel hierarchy , though various applications of multilevel optimization have been already discussed in the 1990s [ 24 ] . As far as we know , few solution methods with a theoretical guarantee have been proposed . A metaheuristic algorithm based on a perturbation , i.e. , a solution method in which the neighborhood is randomly searched for a better solution in the order of x1 to xn has been proposed in [ 23 ] for general n-level optimization problems . The effect of updating xi on the decision variables xi+1 , . . . , xn of the lower-level problems is not considered , and hence , the hierarchical structure of the multilevel optimization problem can not be fully utilized in updating the solution . Because of this , the algorithm does not have a theoretical guarantee for the obtained solution . Very recently ( two days before the NeurIPS 2021 abstract deadline ) , a proximal gradient method based on the fixed-point theory for trilevel optimization problem [ 21 ] has been proposed . This paper has proved convergence to an optimal solution assuming the convexity of the objective functions . However , it does not include numerical results and hence its practical efficiency is unclear . Contribution of this paper In this paper , by extending the gradient method for bilevel optimization problems [ 11 ] to multilevel optimization problems , we propose an algorithm with a convergence guarantee for multilevel optimization other than one by [ 21 ] . Increasing the problem hierarchy from two to three drastically makes developing algorithms and showing theoretical guarantees difficult . In fact , as discussed above , in the case of bilevel , replacing the second-level optimization problem with its optimality condition or replacing it with T steepest-descent sequential updates immediately results in a one-level optimization problem . However , when it comes to n ≥ 3 levels , it may be necessary to perform this replacement n times , and it seems not easy to construct a solution method with a convergence guarantee . It is not straightforward at all to give our method a theoretical guarantee similar to the one [ 11 ] developed for bilevel optimization . We have confirmed the effectiveness of the proposed method by applying it to hyperparameter learning with real data . Experimental verifications of the effectiveness of trilevel optimization using real-world problems are the first ones as far as we know . 2 Related existing methods . 2.1 Two-stage robust optimization problems . When making long-term decisions , it is also necessary to make many decisions according to the situation . If the same objective function is acceptable throughout the period , i.e. , f1 = · · · = fn , and there are hostile players , such a problem is often formulated as a multistage robust optimization problem . Especially , the concept of two-stage robust optimization ( so-called adjustable robust optimization ) minx1∈S1 maxx2∈S2 minx3∈S3 f1 ( x1 , x2 , x3 ) was introduced with feasible sets Si ⊆ Rdi for i = 1 , 2 , 3 , and methodology based on affine decision rules was developed by [ 4 ] . Researches based on affine policies are still being actively conducted ; see , e.g. , [ 6 , 26 ] . The two-stage robust optimization can be considered as a special case of ( 1 ) with n = 3 when constraints x2 ∈ S2 ⊆ Rd2 and x3 ∈ S3 ⊆ Rd3 are added . This approach is unlikely to be applicable to our problem ( 1 ) without strong assumptions such as affine decision rules . There is another research stream on min-max-min problems including integer constraints ( see e.g. , [ 7 , 25 ] ) . They transform the problem into the min-max problem by taking the dual for the inner “ min ” and apply cut-generating approaches or Benders decomposition techniques with the property of integer variables . While the approach is popular in the application studies of electric grid planning , it is restricted to the specific min-max-min problems and no more applicable to multilevel optimization problems ( 1 ) . 2.2 Existing methods for bilevel optimization problems . Various applications are known for bilevel optimization problems , i.e. , the case of n = 2 in ( 1 ) : min x1∈S1 f1 ( x1 , x2 ) s.t . x2 = argmin x′2 f2 ( x1 , x ′ 2 ) . ( 2 ) One of the most well-known applications in machine learning is hyperparameter learning . For example , the problem that finds the best hyperparameter value in the ridge regression is formulated as min λ≥0 ∥yvalid −Xvalidθ∥22 s.t . θ = argmin θ′ ∥ytrain −Xtrainθ′∥22 + λ∥θ′∥22 , where ( Xtrain , ytrain ) are training samples , ( Xvalid , yvalid ) are validation samples , and λ is a hyperparameter that is optimized in this problem . Under Assumption 1 restricted to n = 2 shown later , the existence of an optimal solution to the bilevel problem ( 2 ) is ensured by [ 12 , Theorem 3.1 ] . There are mainly two approaches to solve bilevel optimization problems . The old practice is to replace the lower-level problem with its optimality condition and to solve the resulting single-level problem . It seems difficult to use this approach to multilevel problems because we need to apply the replacement n-times . The other one is to use gradient-based methods developed by Franceschi et al . [ 11 , 12 ] for bilevel optimization problems . Their approach reduces ( 2 ) to a single-level problem by replacing the lower-level problem with T equations using a prechosen number T . Hereinafter , we briefly summarize the results of Franceschi et al . [ 11 , 12 ] for bilevel optimization . Under the continuous differentiability assumption for f2 , x2 is iteratively updated by x ( t ) 2 = Φ ( t ) ( x1 , x ( t−1 ) 2 ) at the tth iteration using an iterative method , e.g. , the gradient descent method : Φ ( t ) ( x1 , x ( t−1 ) 2 ) = x ( t−1 ) 2 − α ( t ) ∇x2f2 ( x1 , x ( t−1 ) 2 ) . Then , the bilevel optimization problem is approximated by the single-optimization problem with T equality constraints and new variables { x ( t ) 2 } Tt=1 instead of x2 : min x1∈S1 , { x ( t ) 2 } f1 ( x1 , x ( T ) 2 ) s.t . x ( t ) 2 = Φ ( t ) ( x1 , x ( t−1 ) 2 ) ( t = 1 , . . . , T ) , ( 3 ) where x ( 0 ) 2 is a given constant . Eliminating x ( 1 ) 2 , . . . , x ( T ) 2 using constraints , we can equivalently recast the problem above into the unconstrained problem minx1∈S1 F̃1 ( x1 ) . Under assumption on differentiability , the gradient of F̃1 ( x1 ) is given in [ 11 , Section 3 ] by ∇x1 F̃1 ( x1 ) = ∇x1f1 ( x1 , x ( T ) 2 ) + T∑ t=1 B ( t ) ( T∏ s=t+1 A ( s ) ) ∇x2f1 ( x1 , x ( T ) 2 ) , A ( t ) = ∇x2Φ ( t ) ( x1 , x ( t−1 ) 2 ) ( t = 1 , . . . , T ) , B ( t ) = ∇x1Φ ( t ) ( x1 , x ( t−1 ) 2 ) ( t = 1 , . . . , T ) . Roughly speaking , [ 12 , Theorem 3.2 ] proved that the optimal value and the solution set of ( 3 ) converge to those of ( 2 ) as T → ∞ under Assumptions 1 and 3 restricted to n = 2 . 3 Multilevel optimization problems and their approximation . We will develop an algorithm for multilevel optimization problems ( 1 ) under some assumptions by extending the studies [ 11 , 12 ] on bilevel problems . Our algorithm and its theoretical guarantee look similar to those in [ 11 , 12 ] , but they are not straightforwardly obtained . As emphasized in Section 1 , unlike the bilevel problem , even if the lower-level problem is replaced with T steepest-descent sequential updates , the resulting problem is still a multilevel problem . In this section , we show how to resolve these difficulties that come from the multilevel hierarchy .

Summary: The authors propose an approximate algorithm for multilevel optimization problems. The presented method can be considered an $n$-level extension of a related and previously published algorithm for bilevel optimization. It relies essentially on replacing optimal solutions in lower-level problems by respective $T$-step gradient descent approximations. That way, the original multilevel problem can be approximated in terms of an unconstrained problem depending only on top-level variables. After an an introduction and discussion of related work, the authors describe how the objective of the approximate problem can be differentiated with respect to the top-level variables. Moreover, assumptions are given under which the original problem admits an optimal solution. And it is shown that the optimal value of the approximate problem converges to a solution of the original problem in the limiting case $T\rightarrow\infty$. Finally, the authors report numerical results obtained through application of their method to two different trilevel optimization problems.