Title: UNIVERSAL REPRESENTATION OF GENERALIZED CONVEX FUNCTIONS AND THEIR GRADIENTS
Primary area: optimization
Keywords: Generalized Convexity, Universal Approximation, Differentiable Layers, Bilevel Optimization, Optimal Transport, Adversarial Training

Abstract: Solutions to a wide range of optimization problems take the form of generalized convex functions (GCFs) or their gradients. These functions are ubiquitous in practice and their structure can be leveraged to collapse certain nested bilevel objectives (i.e. adversarial training) into single-level problems amenable to standard first order optimization methods. In spite of recent advances in parameterizing convex functions and their gradients, the more general GCFs and their gradients have received little attention. We first motivate their study by highlighting their applications, then provide a new differentiable layer with a convex parameter space and show it and its gradient are universal approximators for GCFs and their gradients respectively. Finally, we put the theory to practice by demonstrating its effectiveness in learning optimal pricing mechanisms when selling multiple goods. The methods developed here are implemented in a publicly available supplementary package that also contains the code for reproducing the experiments.

TL;DR: We show how generalized convex functions can be parameterized with universal approximation guarantees, yielding differentiable layers that simplify bilevel optimization and improve applications in adversarial training and optimal transport.