Automatic and Harmless  Regularization  with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach

1 Introduction . Although machine learning ( ML ) has been typically conceptualized as optimizing a single objective function , most practical ML tasks actually involve trading off two or more objective functions , such as the data fitness function vs. a regularization or auxiliary loss . Let f and g be two objectives functions of interest on Rd . A principled way to trade-off f and g is through constrained optimization : min ✓2Rd f ( ✓ ) s.t . g ( ✓ ) c , ( 1 ) where c is a threshold . When varying c , solutions of ( 1 ) cover the Pareto points of ( f , g ) , providing different trade-offs on f , g. Moreover , ( 1 ) naturally arises when g is an important constraint that should be controlled explicitly , e.g. , in terms of safety , fairness , and other trustworthy measures . However , compared to unconstrained optimization , constrained optimization ( 1 ) has been much less widely used in practical machine learning . In fact , perhaps the most common approach to handling ( 1 ) is to transform it into the unconstrained optimization of the linear combination of f and g , min ✓2Rd f ( ✓ ) + g ( ✓ ) , ( 2 ) where the trade-off between f and g is controlled by a weight coefficient , instead of the threshold parameter c. A folklore argument is that can be viewed as the Lagrange multiplier of ( 1 ) , and hence ( 1 ) can be mapped into ( 2 ) with properly selected . Unfortunately , despite being broadly used , ( 2 ) is insufficient to fully replace ( 1 ) and suffers from a number of disadvantages in both theory and practice : 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . • Interpretability . provides a less intuitive parameter to select than c , since it depends on the relative scale of f and g , whose range is problem-dependent and needs to be optimized as a hyperparameter . In comparison , the threshold c can be specified as a tolerance parameter when g is a metric that users want to control explicitly . • Pareto Coverage . When f , g are non-convex functions , ( 1 ) provides a strictly broader class of problems than ( 2 ) , because for some c , there may exist no 2 R , such that ( 1 ) and ( 2 ) are equivalent . From the multi-objective optimization ( MOO ) perspective , this is related to the fact that ( 2 ) can only capture the convex envelop of the Pareto front while varying , while ( 1 ) provides all Pareto optimal points by varying c. • Invariance . The constrained optimization ( 1 ) is invariant to arbitrary monotonically increasing maps on f and g. That is , let and 0 be two monotonically increasing maps , then we obtain the equivalent problem if we replace f with f , and g with 0 g , and c with 0 ( c ) . In comparison , applying nonlinear transforms on f and g in ( 2 ) yields fundamentally different problems . • Harmless Regularization . In many practical cases , one of the objectives ( say f ) is of secondary importance w.r.t . the other one ( say g ) , in sense that we are interested in minimizing f only when g is fully optimized ; this can be formulated as the following lexicographic optimization problem : min ✓2Rd f ( x ) s.t . g ( ✓ ) g⇤ where g⇤ : = inf ✓2Rd g ( ✓ ) , ( 3 ) where we minimize f inside the optimum set { ✓ : g ( ✓ ) g⇤ } of g. This can be viewed as a special case of ( 1 ) with the minimum threshold c = g⇤ ( which is unknown before hand ) , and can not be captured by ( 2 ) with a finite and fixed . Compared with constrained optimization ( 1 ) , the lexicographic ( or simply lexico ) problem eliminates the need of setting the threshold c , and hence provides a handy , coefficient-free approach for incorporating auxiliary losses without hurting the main loss . As our empirical results show , this finds useful in numerous deep learning tasks , e.g. , fairness ML and semi-supervised learning . Our Contributions We conduct a joint study on the constrained and lexico optimization problems . We propose a simple and general local descent algorithm ( Algorithm 1 ) which offers a unified solution to both problems . By placing a dynamic barrier constraint on the search direction at each iteration , the algorithm finds a trajectory towards the optimal solution by properly balancing f and g with an adaptive combination coefficient coefficient t ; as seen in Eq 4 , the t is decided with a simple formula by the inner prod uct between the objective and constraint gradientsrf and rg . In Section 3.1-3.3 , we study the continuous-time convergence of the method for general non-convex functions and both the constrained and lexico optimization cases . In Section 3.4 , we elaborate the ( often overlooked ) fact that methods based on optimizing the linear combination ( 2 ) is fundamentally unsound as an approach to constrained optimization ( 1 ) with non-convex functions and non-zero duality gap . In Section 4 , we empirically show that our method provides an efficient approach to approximating Pareto sets and incorporating side information in a variety of deep learning tasks . Algorithm 1 Dynamic Barrier Gradient Descent for ( 1 ) and ( 3 ) Choose stepsize { ✏t } and the dynamic barrier function in ( 8 ) ( use ↵ = = 1 by default ; set ĝ = c for constrained optimization ( 1 ) and ĝ to be any value no larger than g⇤ for lexicographic optimization ( 3 ) ) . for iteration t do ✓t+1 ✓t ✏t ( rf ( ✓t ) + trg ( ✓t ) ) , t = max ( ✓t ) rf ( ✓t ) > rg ( ✓t ) krg ( ✓t ) k2 , 0 ! . ( 4 ) end for 2 Main Method . We introduce the main algorithm 1 , a simple gradient-based method for solving both constrained optimization ( 1 ) and lexicographic optimization ( 3 ) in a unified way . The method performs iterative updates of form ✓t+1 ✓t ✏tvt , ( 5 ) where ✏t 0 is a step size and vt 2 Rd is an update direction to be chosen to balance the minimization of f and constraint satisfaction on g. The vt is designed to satisfy the following desiderata : 1 ) When the constraint is not satisfied ( i.e. , g ( ✓t ) > c ) , we should mainly focus on decreasing g to meet the constraint as fast as possible ; meanwhile , f should act as a secondary objective in this phase , meaning that f should be minimized upto the degree that it does not hurt the descent of g. 2 ) When the constraint is met ( i.e. , g ( ✓t ) c ) , we should prioritize to minimize f , which is made possible in general only if we allow g to increase . However , the increasing rate of g should be properly controlled , so that ✓ stays inside or nearby the feasible set while we minimize f . It turns out that both properties can be achieved if we select vt by the following optimization : vt = argmin v2Rd n krf ( ✓t ) vk2 s.t . rg ( ✓t ) > v ( ✓t ) o , ( 6 ) where we want vt to be as close to rf ( ✓t ) as much as possible ( and hence decrease f ) , but subject to a lower bound on the inner product of rg ( ✓t ) and vt to ensure that the change of g is properly controlled by the location of ✓t ; here the lower bound : Rd ! R is a dynamic barrier function which trade-offs loss minimization with constraint satisfaction by controlling the inner product between rg ( ✓t ) and vt. To achieve the desiderata on vt , we should let ( ✓t ) have the same sign as g ( ✓t ) c , so that the constraint { ✓ : g ( ✓ ) c } is equivalent to { ✓ : ( ✓ ) 0 } , that is , sign ( ( ✓ ) ) = sign ( g ( ✓ ) c ) , ( 7 ) where sign ( x ) = x|x| for x 6= 0 and sign ( 0 ) = 0 . In this way , when the step size ✏t is sufficiently small , we have the following properties that will be studied theoretically in Section 3 : 1 ) When ✓t is outside of the feasible set ( g ( ✓t ) > c ) , the constraint is rg ( ✓t ) > vt ( ✓t ) > 0 , which ensures that g decreases strictly outside of the feasible set . 2 ) When ✓t is on the boundary ( g ( ✓t ) = c ) , the constrain reduces to rg ( ✓t ) > vt ( ✓t ) = 0 , meaning that we want to decrease f subject to that g does not increase . 3 ) When ✓t is in the interior of the feasible set ( g ( ✓t ) < c ) , the lower bound on rg ( ✓t ) > vt is negative , allowing g to increase . In this case , the optimal solution vt in ( 6 ) can be shown to have a positive inner product with rf ( ✓t ) unless rf ( ✓t ) = 0 , which means that the algorithm monotonically decreases f until a local optimum is reached . Computing t Since ( 6 ) is a simple quadratic convex programming , it is easy to see that the solution is vt = rf ( ✓t ) + trg ( ✓t ) where t solves the following dual problem of ( 6 ) : t = argmin 0 n krf ( ✓t ) + rg ( ✓t ) k2 ( ✓t ) o = max ( ✓t ) rf ( ✓t ) > rg ( ✓t ) krg ( ✓t ) k2 , 0 ! . Practical Choice of There are a broad range of ’ s that satisfy ( 7 ) . Two particularly simple choices stand out which are suitable for constrained and lexico optimization , respectively : 1 ) ( ✓t ) = ↵ ( g ( ✓t ) c ) , where ↵ > 0 . This puts a strong requirement on descending the constraint g when g ( ✓t ) c is large and positive , and it gives higher flexibility for minimizing f when g ( ✓t ) c is negative as we move towards the interior of the feasible set . In this case , the inner product constraint in ( 6 ) can be viewed as a linearization of the original constraint ( g ( ✓ ) c ) in ( 1 ) : g ( ✓t ✏tvt ) ⇡ g ( ✓t ) ✏trg ( ✓t ) > vt c , with ✏t = 1/↵ . Hence , the algorithm in this case can be viewed as a simple variant of sequential quadratic programming ( SQP ) ( e.g. , Nocedal & Wright , 2006 ) when we use an identity matrix to approximate the Hessian of f . 2 ) ( ✓t ) = krg ( ✓t ) k2 , where > 0 . Assume that rg ( ✓t ) = 0 implies global minimum of g , then this satisfies the sign condition ( 7 ) for the lexico case ( 3 ) , without requiring to estimate the unknown threshold g⇤ . Because ( ✓t ) 0 in this case , it allows us to strictly decrease g until ✓t reaches an optimum of g. Meanwhile , f acts as the secondary objective that is decreased when it does not interfere with the descent of g. In practice , when g has multiple local minima , then the algorithm would minimize f ( ✓ ) subject to g ( ✓ ) g ( ✓̂⇤ ) , where ✓̂⇤ is a local minimum of g that the algorithm encounters . Combining the two cases into a single , we propose to use ( ✓ ) = min ⇣ ↵ ( g ( ✓ ) ĝ ) , krg ( ✓ ) k2 ⌘ , ( 8 ) where we take ĝ = c if the goal is to solve the constrained optimization ( 1 ) with threshold c , and take ĝ to be any lower bound of g⇤ ( i.e. , ĝ g⇤ ) for solving the lexicographic optimization problem ( 3 ) ; for example , if g is a norm or mean square loss , then ĝ = 0 is a natural estimation of the lower bound . If no lower bound can be estimated , we can take ĝ = 1 , and ( 8 ) reduces back to ( ✓ ) = krg ( ✓ ) k2 . The choice of the hyper-parameters ↵ , controls the speed of constraint satisfaction vs. loss minimization . In our experiments , we find that it is sufficient to simply set ↵ = = 1 in most cases so that the algorithm introduces no extra hyper-parameters compared with vanilla gradient descent . See Algorithm 1 for a summary of the algorithm procedure . Profiling Pareto Set The Pareto set of ( f , g ) is the set of points ✓ for which there exists no ✓0 , such that f ( ✓0 ) f ( ✓ ) , g ( ✓0 ) g ( ✓ ) , and ( f ( ✓0 ) , g ( ✓0 ) ) 6= ( f ( ✓ ) , g ( ✓ ) ) . The combination of the constrained and lexico optimization provides an automatic approach to obtain uniformly distributed points from the Pareto set of ( f , g ) . To do so , we start with solving the lexico problem ( 3 ) , which corresponds to one of the endpoints of the Pareto set ; we then walk through the Pareto points by solving the constrained optimization ( 1 ) with c increasing linearly from the g⇤ estimated from lexico optimization . Optionally , we can also solve the mirror version min✓ g ( ✓ ) s.t . f ( ✓ ) f⇤ : = inf✓0 f ( ✓0 ) to obtain the other end point , and then solve ( 1 ) with c on a grid between the two end points .

Summary: This work studies constrained and also lexicographic optimization problem. The authors first propose a general local descent algorithm which places a dynamic barrier constraint on the search direction. The authors then investigate the theoretical properties of their algorithm and claim that their algorithm is guaranteed to converge to a local optima for both constrained and lexico optimization with non-convex functions. Finally, the authors conduct experiments to evaluate the performance of their method on several real-world deep learning tasks.