In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable. This problem includes difference of convex (DC) functions and a family of bi-convex functions as special cases. We develop stochastic algorithms and establish their first-order convergence for finding a (nearly) stationary solution of the target non-convex function under different conditions of the component functions. To the best of our knowledge, this is the first work that comprehensively studies stochastic optimization of non-convex inf-projection minimization problems with provable convergence guarantee. Our algorithms enable efficient stochastic optimization of a family of non-decomposable DC functions and a family of bi-convex functions. To demonstrate the power of the proposed algorithms we consider an important application in variance-based regularization, and experiments verify the effectiveness of our inf-projection based formulation and the proposed stochastic algorithm in comparison with previous stochastic algorithms based on the min-max formulation for achieving the same effect.
Recently, @cite_17 proposed a min-max formulation based on distributionally robust optimization for variance-based regularization as following: where @math is a hyper-parameter, @math , @math , and @math is called the @math -divergence based on @math . The above problem is convex-concave when the loss function @math is convex in terms of @math . It is was shown in that the above min-max formulation is equivalent to the problem ) with a proper value of @math with high probability under the assumption that the number of training examples @math is sufficiently large (see Theorem 1 and Theorem 2 in @cite_17 ).
@cite_17
2944407464
In this paper the shortest path and the minimum spanning tree problems in a graph with @math nodes and @math cost scenarios (objectives) are discussed. In order to choose a solution the min-max criterion is applied. The minmax versions of both problems are hard to approximate within @math for any @math . The best approximation algorithm for the min-max shortest path problem, known to date, has approximation ratio of @math . On the other hand, for the min-max spanning tree, there is a randomized algorithm with approximation ratio of @math . In this paper a deterministic @math -approximation algorithm for min-max shortest path is constructed. For min-max spanning tree a deterministic @math -approximation algorithm is proposed, which works for a large class of graphs and a randomized @math -approximation algorithm, which can be applied to all graphs, is constructed. It is also shown that the approximation ratios obtained are close to the integrality gaps of the corresponding LP relaxations. Convex and submodular functions play an important role in many applications, and in particular in combinatorial optimization. Here we study two special cases: convexity in one dimension and submodularity in two dimensions. The latter type of functions are equivalent to the well-known Monge matrices. A matrix @math is called a Monge matrix if for every @math and @math we have @math . If inequality holds in the opposite direction, then V is an inverse Monge matrix (supermodular function). Many problems, such as the traveling salesperson problem and various transportation problems, can be solved more efficiently if the input is a Monge matrix. In this work we present testing algorithms for the above properties. A testing algorithm for a predetermined property @math is given query access to an unknown function f and a distance parameter @math . The algorithm should accept f with high probability if it has the property @math and reject it with high probability if more than an @math -fraction of the function values should be modified so that f obtains the property. Our algorithm for testing whether a 1-dimensional function @math is convex (concave) has query complexity and running time of @math . Our algorithm for testing whether an n1 × n2 matrix V is a Monge (inverse Monge) matrix has query complexity and running time of @math . We investigate the minimization of Newton's functional for the problem of the body of minimal resistance with maximal height @math cite butt in the class of convex developable functions defined in a disc. This class is a natural candidate to find a (non-radial) minimizer in accordance with the results of cite lrp2. We prove that the minimizer in this class has a minimal set in the form of a regular polygon with @math sides centered in the disc, and numerical experiments indicate that the natural number @math is a non-decreasing function of @math . The corresponding functions all achieve a lower value of the functional than the optimal radially symmetric function with the same height @math . Let @math be a finite lattice and let @math be a positive integer. A function @math is said to be submodular if @math for all @math . In this paper we study submodular functions when @math is a diamond. Given oracle access to @math we are interested in finding @math such that @math as efficiently as possible. We establish a min--max theorem, which states that the minimum of the submodular function is equal to the maximum of a certain function defined over a certain polyhedron; and a good characterisation of the minimisation problem, i.e., we show that given an oracle for computing a submodular @math and an integer @math such that @math , there is a proof of this fact which can be verified in time polynomial in @math and @math ; and a pseudo-polynomial time algorithm for the minimisation problem, i.e., given an oracle for computing a submodular @math one can find @math in time bounded by a polynomial in @math and @math .