Surrogate-Based Constrained Langevin Sampling With Applications to Optimal Material Configuration Design

1 INTRODUCTION . In many real-world design problems , the optimal design needs to simultaneously satisfy multiple constraints , which can be expensive to estimate . For example , in computational material design , the goal is to come up with material configurations , or samples , satisfying a list of physical constraints that are given by black-box numerical Partial Differential Equations ( PDE ) solvers . Such solvers ( for example , the Boltzmann Transport Equation solver ) are often complex , expensive to evaluate , and offer no access to their inner variables or their gradients . We pose this design-under-constraints problem as sampling from a Gibbs distribution defined on some compact support . The problem of sampling from a distribution with unknown likelihood that can only be point-wise evaluated is called black-box sampling ( Chen & Schmeiser , 1998 ; Neal , 2003 ) . We show in this paper that constrained black-box sampling can be cast as a constrained Langevin dynamics with gradient-free methods . Zero-order optimization via Gaussian smoothing was introduced in Nesterov & Spokoiny ( 2017 ) and extended to black-box sampling with Langevin dynamics in Shen et al . ( 2019 ) . We extend this approach to the constrained setting from a black-box density with compact support . However , one shortcoming of this approach is that it is computationally very expensive since it requires repeatedly querying PDE solvers in order to get an estimate of the gradient . To alleviate computational issues , we propose Surrogate Model Based Langevin dynamics , that consists of two steps : ( i ) Learning ( using training data ) an approximation of the gradient of the potential of the Gibbs distribution . We show that learning the gradient , rather than the potential itself , is important for the mixing of the Langevin dynamics towards the target Gibbs distribution . We devise several objective functions , as well as deep neural-network architectures for parameterizing the approximating function class , for learning the gradient of the potential function . ( ii ) We then use the surrogate gradient model in the constrained Langevin dynamics in lieu of the black-box potential . Using the surrogate enables more efficient sampling , since it avoids querying the expensive PDE solvers , and obtaining gradients is as efficient as evaluating the functions themselves using automatic differentiation frameworks such as PyTorch or TensorFlow . To summarize , our main contributions are as follows : 1 . We cast the problem of generating samples under constraints in the black-box setting as sampling from a Gibbs distribution . 2 . We introduce Constrained Zero-Order Langevin Monte Carlo , using projection or proximal methods , and provide the proof of its convergence to the target Gibbs distribution . 3 . We introduce Surrogate Model Based Projected Langevin Monte Carlo via learning the gradient of the potential of the Gibbs distribution using deep neural networks or reproducing kernel spaces , and prove its convergence to the target distribution when used in conjunction with projection or proximal based methods . We shed the light on the importance of the approximation of the gradient of the potential , and we show how to achieve this using Hermite and Taylor learning . 4 . We showcase the usability and effectiveness of the proposed methods for the design of nanoporous configurations with improved thermoelectric efficiency . The design consists of finding new configurations with optimized pore locations , such that the resulting configurations have favorable thermal conductivity ( i.e. , minimal κ ) and desired mechanical stability ( von Mises Stress σ ≤ τ , where τ is some preset threshold ) . 1 2 FROM CONSTRAINTS SATISFACTION TO SAMPLING FROM A GIBBS DISTRIBUTION : POSTERIOR REGULARIZATION . In black-box optimization problems ( such as the material design under consideration ) , the goal is to find a posterior distribution q of samples satisfying a list of equality and inequality constraints : ψj ( x ) = yk , j = 1 . . . Ce , and φk ( x ) ≤ bk , k = 1 . . . Ci where x ∈ Ω and Ω ⊂ Rd is a bounded domain . We assume a prior distribution p0 ( whose analytical form is known ) . The main challenge in black-box optimization is that the functions ψj and φk can be only evaluated point-wise , and neither do we have functional forms nor access to their gradients . For example , ψ and φ might be obtained via aggregating some statistics on the solution of a nonlinear PDE given by a complex solver . To make the problem of learning under constraints tractable , we choose Lagrangian parameters λj > 0 and obtain the following relaxed objective : min q , ∫ Ω q ( x ) =1 KL ( q , p0 ) + Ce∑ j=1 λjEx∼q ( ψj ( x ) − yk ) 2 + Ci∑ k=1 λkEx∼q ( φk ( x ) − bk ) + ( 1 ) The formulation in Eq . 1 is similar in spirit to the posterior regularization framework of Ganchev et al . ( 2010 ) ; Hu et al . ( 2018 ) . However , we highlight two differences : ( i ) our focus is on constrained settings ( where Ω is bounded ) , and ( ii ) we assume a black-box setting . We first obtain : Lemma 1 ( Constraint Satisfaction as Sampling from a Gibbs Distribution ) . The solution to the distribution learning problem given in Eq . 1 is given by : π ( x ) = exp ( −U ( x ) ) Z 1x∈Ω ( 2 ) where U ( x ) = − log p0 ( x ) + ∑Ce j=1 λj ( ψj ( x ) − yk ) 2 + ∑Ci k=1 λk ( φk ( x ) − bk ) + and Z =∫ x∈Ω exp ( −U ( x ) ) dx . Lemma 1 shows that the constraint satisfaction problem formulated in Eq . 1 amounts to sampling from a Gibbs distribution defined on a compact support given in Eq . 2 . Sampling from a Gibbs 1Note that both properties κ and σ for a given configuration are obtained by numerically solving highly non-linear PDEs . The material configuration is defined by the pore locations , the material used , and the response of the material to heat ( thermal ) or stress ( mechanical ) flows . distribution ( also known as Boltzmann distribution ) has a long history using Langevin dynamics . In the white-box setting when the functions defining the constraints have explicit analytical forms as well as their gradients , Langevin dynamics for Gibbs distribution sampling defined on a compact domain Ω and their mixing properties were actively studied in Bubeck et al . ( 2015 ) ; Brosse et al . ( 2017 ) . In the next Section , we provide a more detailed review . Remark 1 ( Relation to Bayesian Optimization ) . While in Bayesian optimization we are interested in finding a point that satisfies the constraints , in our setting we are interested in finding a distribution of candidate samples that satisfy ( black-box ) constraints . See ( Suzuki et al. , 2019 ) for more details . Remark 2 . For the rest of the paper , we will assume p0 to be the uniform distribution on Ω , which means that its gradients are zero on the support of the domain Ω . Otherwise , if p0 is known and belongs to , for instance , an exponential family or a generative model prior ( such as normalizing flows ) , we can sample from π using a mixture of black-box sampling on the constraints ( ψj , φk ) and white-box sampling on log ( p0 ) . 3 WHITE-BOX SAMPLING : CONSTRAINED LANGEVIN DYNAMICS . We review in this section Langevin dynamics in the unconstrained case ( Ω = Rd ) and the constrained setting ( Ω ⊂ Rd ) . Below , ‖·‖ denotes the Euclidean norm unless otherwise specified . We are interested in sampling from π ( x ) = 1 Z exp ( −U ( x ) ) 1x∈Ω , ( 3 ) Preliminaries . We give here assumptions , definitions and few preliminary known facts that will be useful later . Those assumptions are commonly used in Langevin sampling analysis ( Dalalyan , 2017 ; Bubeck et al. , 2015 ; Brosse et al. , 2017 ; Durmus et al. , 2019 ) . 1 . Assumption A : We assume Ω is a convex such that 0 ∈ Ω , Ω contains a Euclidean ball of radius r , and Ω is contained in a Euclidean ball of radius R. ( For example , Ω might encode box constraints . ) The projection onto Ω , PΩ ( x ) is defined as follows : for all x ∈ Ω , PΩ ( x ) =arg minz∈Ω ‖x− z‖ 2 . Let R = supx , x′∈Ω ||x− x′|| < ∞ . 2 . Assumption B : We assume that U is convex , β-smooth , and with bounded gradients : ‖∇xU ( x ) −∇yU ( y ) ‖ ≤ β ‖x− y‖ , ∀x , y ∈ Ω ( β-smoothness ) . ‖∇U ( x ) ‖ ≤ L , ∀x ∈ Ω ( Boundedness ) . The Total Variation ( TV ) distance between two measures µ , ν is defined as follows : TV ( µ , ν ) = supA |µ ( A ) − ν ( A ) | . Pinsker Inequality relates KL divergence to TV : TV ( µ , ν ) ≤ √ 2KL ( µ , ν ) . Unconstrained Langevin Dynamics . In the unconstrained case , the goal is to sample from a Gibbs distribution π ( x ) = exp ( −U ( x ) ) /Z that has unbounded support . This sampling can be done via the Langevin Monte Carlo ( LMC ) algorithm , which is given by the following iteration : Xk+1 = Xk − η∇xU ( Xk ) + √ 2ληξk , k = 0 . . .K − 1 ( LMC ) , ( 4 ) where ξk ∼ N ( 0 , Id ) , η is the learning rate , and λ > 0 is a variance term . Constrained Langevin Dynamics . In the constrained case , the goal is to sample from π ( x ) = exp ( −U ( x ) ) /Z1x∈Ω , . We discuss two variants : Projected Langevin Dynamics . Similar to projected gradient descent , Bubeck et al . ( 2015 ) introduced Projected Langevin Monte Carlo ( PLMC ) and proved its mixing propreties towards the stationary distribution π. PLMC is given by the following iteration : Xk+1 = PΩ ( Xk − η∇xU ( Xk ) + √ 2ληξk ) , k = 0 . . .K − 1 ( PLMC ) , ( 5 ) In essence , PLMC consists of a single iteration of LMC , followed by a projection on the set Ω using the operator PΩ . Proximal Langevin Dynamics . Similar to proximal methods in constrained optimization , Brosse et al . ( 2017 ) introduced Proximal LMC ( ProxLMC ) that uses the iteration : Xk+1 = ( 1− η γ ) Xk−η∇xU ( Xk ) + η γ PΩ ( Xk ) + √ 2ληξk , k = 0 . . .K−1 , ( ProxLMC ) ( 6 ) where η is the step size and γ is a regularization parameter . In essence , ProxLMC ( Brosse et al. , 2017 ) performs an ordinary LMC on Uγ ( x ) = U ( x ) + iγΩ ( x ) , where i γ Ω ( x ) is the proximal operator : iγΩ ( x ) = infy iΩ ( x ) + ( 2γ ) −1 ‖x− y‖2 = ( 2γ ) −1 ‖x− PΩ ( x ) ‖2 , where iΩ ( x ) = 0 for x ∈ Ω and iΩ ( x ) = ∞ for x /∈ Ω . Therefore , the update in Eq . 6 is a regular Langevin update ( as in Eq . 4 ) with potential gradient∇xUγ ( x ) = ∇xU ( x ) + γ−1 ( x− PΩ ( x ) ) . We denote by µPLMCK and µ ProxLMC K the distributions of XK obtained by iterating Eq . 5 and Eq . 6 respectively . Under Assumptions A and B , both these distributions converge to the target Gibbs distribution π in the total variation distance . In particular , Bubeck et al . ( 2015 ) showed that for η = Θ̃ ( R2/K ) , we obtain : TV ( µPLMCK , π ) ≤ ε for K = Ω̃ ( ε−12d12 ) . ( 7 ) Likewise , Brosse et al . ( 2017 ) showed that for 0 < η ≤ γ ( 1 + β2γ2 ) −1 , we obtain : TV ( µProxLMCK , π ) ≤ ε for K = Ω̃ ( ε−6d5 ) , ( 8 ) where the notation αn = Ω̃ ( βn ) means that there exists c ∈ R , C > 0 such that αn ≥ Cβn logc ( βn ) .

Summary: The paper considers the problem of sampling points from a constrained set in R^d where the constraints can only be accessed in a zero order fashion. They consider the specific situation where the constraints are a solution of a complicated PDE solver and hence the derivatives or specific functional forms of the constraint cannot be obtained. They repose the problem as sampling from a Gibbs distribution whose potential contains constraints as penalties in a Lagrangian fashion. They now wish to sample from the Gibbs distribution using Langevin diffusion.  The Langevin process requires a derivative of the gradient. The setting does not allow for that and therefore the authors propose two approaches -