DC3: A learning method for optimization with hard constraints

1 INTRODUCTION . Traditional approaches to constrained optimization are often expensive to run for large problems , necessitating the use of function approximators . Neural networks are highly expressive and fast to run , making them ideal as function approximators . However , while deep learning has proven its power for unconstrained problem settings , it has struggled to perform well in domains where it is necessary to satisfy hard constraints at test time . For example , in power systems , weather and climate models , materials science , and many other areas , data follows well-known physical laws , and violation of these laws can lead to answers that are unhelpful or even nonsensical . There is thus a need for fast neural network approximators that can operate in settings where traditional optimizers are slow ( such as non-convex optimization ) , yet where strict feasibility criteria must be satisfied . In this work , we introduce Deep Constraint Completion and Correction ( DC3 ) , a framework for applying deep learning to optimization problems with hard constraints . Our approach embeds differentiable operations into the training of the neural network to ensure feasibility . Specifically , the network outputs a partial set of variables with codimension equal to the number of equality constraints , and “ completes ” this partial set into a full solution . This completion process guarantees feasibility with respect to the equality constraints and is differentiable ( either explicitly , or via the implicit function theorem ) . We then fix any violations of the inequality constraints via a differentiable correction procedure based on gradient descent . Together , this process of completion and correction enables feasibility with respect to all constraints . Further , this process is fully differentiable and can be incorporated into standard deep learning methods . Our key contributions are : • Framework for incorporating hard constraints . We describe a general framework , DC3 , for incorporating ( potentially non-convex ) equality and inequality constraints into deeplearning-based optimization algorithms . • Practical demonstration of feasibility . We implement the DC3 algorithm in both convex and non-convex optimization settings . We demonstrate the success of the algorithm in producing approximate solutions with significantly better feasibility than other deep learning approaches , while maintaining near-optimality of the solution . ∗These authors contributed equally . • AC optimal power flow . We show how the general DC3 framework can be used to optimize power flows on the electrical grid . This difficult non-convex optimization task must be solved at scale and is especially critical for renewable energy adoption . Our results greatly improve upon the performance of general-purpose deep learning methods on this task . 2 RELATED WORK . Our approach is situated within the broader literature on fast optimization methods , and draws inspiration from literature on implicit layers and on incorporating constraints into neural networks . We briefly describe each of these areas and their relationship to the present work . Fast optimization methods . Many classical optimization methods have been proposed to improve the practical efficiency of solving optimization problems . These include general techniques such as constraint and variable elimination ( i.e. , the removal of non-active constraints or redundant variables , respectively ) , as well as problem-specific techniques ( e.g. , KKT factorization techniques in the case of convex quadratic programs ) ( Nocedal & Wright , 2006 ) . Our present work builds upon aspects of this literature , applying concepts from variable elimination to reduce the number of degrees of freedom associated with the optimization problems we wish to solve . In addition to the classical optimization literature , there has been a large body of literature in deep learning that has sought to approximate or speed up optimization models . As described in reviews on topics such as combinatorial optimization ( Bengio et al. , 2020 ) and optimal power flow ( Hasan et al. , 2020 ) , ML methods to speed up optimization models have thus far taken two main approaches . The first class of approaches , akin to work on surrogate modeling ( Koziel & Leifsson , 2013 ) , has involved training machine learning models to map directly from optimization inputs to full solutions . However , such approaches have often struggled to produce solutions that are both feasible and ( near- ) optimal . The second class of approaches has instead focused on employing machine learning approaches alongside or in the loop of optimization models , e.g. , to learn warm-start points ( see , e.g. , Baker ( 2019 ) and Dong et al . ( 2020 ) ) or to enable constraint elimination techniques by predicting active constraints ( see , e.g. , Misra et al . ( 2018 ) ) . We view our work as part of the former set of approaches , but drawing important inspiration from the latter : that employing structural knowledge about the optimization model is paramount to achieving both feasibility and optimality . Constraints in neural networks . While deep learning is often thought of as wholly unconstrained , in reality , it is quite common to incorporate ( simple ) constraints within deep learning procedures . For instance , softmax layers encode simplex constraints , sigmoids instantiate upper and lower bounds , ReLUs encode projections onto the positive orthant , and convolutional layers enforce translational equivariance ( an idea taken further in general group-equivariant networks ( Cohen & Welling , 2016 ) ) . Recent work has also focused on embedding specialized kinds of constraints into neural networks , such as conservation of energy ( see , e.g. , Greydanus et al . ( 2019 ) and Beucler et al . ( 2019 ) ) , and homogeneous linear inequality constraints ( Frerix et al. , 2020 ) . However , while these represent common “ special cases , ” there has to date been little work on building more general hard constraints into deep learning models . Implicit layers . In recent years , there has been a great deal of interest in creating structured neural network layers that define implicit relationships between their inputs and outputs . For instance , such layers have been created for SAT solving ( Wang et al. , 2019 ) , ordinary differential equations ( Chen et al. , 2018 ) , normal and extensive-form games ( Ling et al. , 2018 ) , rigid-body physics ( de Avila Belbute-Peres et al. , 2018 ) , sequence modeling ( Bai et al. , 2019 ) , and various classes of optimization problems ( Amos & Kolter , 2017 ; Donti et al. , 2017 ; Djolonga & Krause , 2017 ; Tschiatschek et al. , 2018 ; Wilder et al. , 2018 ; Gould et al. , 2019 ) . ( Interestingly , softmax , sigmoid , and ReLU layers can also be viewed as implicit layers ( Amos , 2019 ) , though in practice it is more efficient to use their explicit form . ) In principle , such approaches could be used to directly enforce constraints within neural network settings , e.g. , by projecting neural network outputs onto a constraint set using quadratic programming layers ( Amos & Kolter , 2017 ) in the case of linear constraints , or convex optimization layers ( Agrawal et al. , 2019 ) in the case of general convex constraints . However , given the computational expense of such optimization layers , these projection-based approaches are likely to be inefficient . Instead , our approach leverages insights from this line of work by using implicit differentiation to backpropagate through the “ completion ” of equality constraints in cases where these constraints can not be solved explicitly ( such as in AC optimal power flow ) . 3 DC3 : DEEP CONSTRAINT COMPLETION AND CORRECTION . In this work , we consider solving families of optimization problems for which the objectives and/or constraints vary across instances . Formally , let x ∈ Rd denote the problem data , and y ∈ Rn denote the solution of the corresponding optimization problem ( where y depends on x ) . For any given x , our aim is then to find y solving : minimize y∈Rn fx ( y ) , s. t. gx ( y ) ≤ 0 , hx ( y ) = 0 , ( 1 ) ( where f , g , and h are potentially nonlinear and non-convex ) . Solving such a family of optimization problems can be framed as a learning problem , where an algorithm must predict an optimal y from the problem data x . We consider deep learning approaches to this task – that is , training a neural network Nθ , parameterized by θ , to approximate y given x . A naive deep learning approach to approximating such a problem involves viewing the constraints as a form of regularization . That is , for training examples x ( i ) , the algorithm learns to minimize a composite loss containing both the objective and two “ soft loss ” terms representing violations of the equality and inequality constraints ( for some λg , λh > 0 ) : ` soft ( ŷ ) = fx ( ŷ ) + λg‖ReLU ( gx ( ŷ ) ) ‖22 + λh‖hx ( ŷ ) ‖22 . ( 2 ) An alternative framework ( see , e.g. , Zamzam & Baker ( 2019 ) ) is to use supervised learning on examples ( x ( i ) , y ( i ) ) for which an optimum y ( i ) is known . In this case , the loss is simply ||ŷ−y ( i ) ||22 . However , both these procedures for training a neural network can lead in practice to highly infeasible outputs ( as we demonstrate in our experiments ) , because they do not strictly enforce constraints . Supervised methods also require constructing a training set ( e.g. , via an exact solver ) , a sometimes difficult or expensive step . To address these challenges , we introduce the method of Deep Constraint Completion and Correction ( DC3 ) , which allows hard constraints to be integrated into the training of neural networks . This method is able to train directly from the problem specification ( instead of a supervised dataset ) , via the following two innovations : Equality completion . We provide a mechanism to enforce equality constraints during training and testing , inspired by the literature on variable elimination . Specifically , rather than outputting the full-dimensional optimization solution directly , we first output a subset of the variables , and then infer the remaining variables via the equality constraints – either explicitly , or by solving an implicit set of equations ( through which we can then backpropagate via the implicit function theorem ) . Inequality correction . We correct for violation of the inequality constraints by mapping infeasible points to feasible points using an internal gradient descent procedure during training . This allows us to fix inequality violations while taking steps along the manifold of points satisfying the equalities , which yields an output that is feasible with respect to all constraints . Overall , our algorithm involves training a neural network Nθ ( x ) to output a partial set of variables z . These variables are then completed to a full set of variables ỹ satisfying the equality constraints . In turn , ỹ is corrected to ŷ to satisfy the inequality constraints while continuing to satisfy the equality constraints . The overall network is trained using backpropagation on the soft loss described in Equation ( 2 ) ( which is necessary for correction , as noted below ) . Algorithm 1 Deep Constraint Completion and Correction ( DC3 ) 1 : assume equality completion procedure ϕx : Rm → Rn−m // to solve equality constraints 2 : 3 : procedure TRAIN ( X ) 4 : init neural network Nθ : Rd → Rm 5 : while not converged do for x ∈ X 6 : compute partial set of variables z = Nθ ( x ) 7 : complete to full set of variables ỹ = [ zT ϕx ( z ) T ] T ∈ Rn 8 : correct to feasible ( or approx . feasible ) solution ŷ = ρ ( ttrain ) x ( ỹ ) 9 : compute constraint-regularized loss ` soft ( ŷ ) 10 : update θ using∇θ ` soft ( ŷ ) 11 : end while 12 : end procedure 13 : 14 : procedure TEST ( x , Nθ ) 15 : compute partial set of variables z = Nθ ( x ) 16 : complete to full set of variables ỹ = [ zT ϕx ( z ) T ] T 17 : correct to feasible solution ŷ = ρ ( ttest ) ( ỹ ) 18 : return ŷ 19 : end procedure We note that as this procedure is somewhat general , in cases where constraints have a specialized structure , specialized techniques may be more appropriate to use . For instance , while we examine linearly-constrained settings in our experiments for the purposes of illustration , in practice , techniques such as Minkowski-Weyl decomposition or Cholesky factorization ( see Frerix et al . ( 2020 ) , Amos & Kolter ( 2017 ) ) may be more efficient in these settings . However , for more general settings without this kind of structure – e.g. , non-convex problems such as AC optimal power flow , which we examine in our experiments – the DC3 framework can provide a ( differentiable ) mechanism for satisfying hard constraints . We now detail the completion and correction procedures used in DC3 .

Summary: This paper proposes a method to strictly enforce hard constraints during a neural network, without compromising differentiability. The method has two stages 1) From a smaller set of predicted variables, compute the remaining ones so that equality constraints are satisfied; 2) Take a few gradient steps (w.r.t soft constraint) in case inequality constraints are violated. They perform experiments on synthetic and also somewhat applied instances of quadratic programs. The results look very promising.