From Data to Knowledge - 402 - Hua Ouyang
Channel: ckleinastro
Date: 10/05/2012

Slides: http://lyra.berkeley.edu/CDIConf/pdfs/Hua_Ouyang_stream_ansgd_slides.pdf

Hua Ouyang: "Stochastic Smoothing for Nonsmooth Minimizations: Accelerating SGD by Exploiting Structure". A video from the UC Berkeley Conference: From Data to Knowledge: Machine-Learning with Real-time and Streaming Applications (May 7-11, 2012).

Abstract
Hua Ouyang (Georgia Institute of Technology)
In this work we consider the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We propose a novel algorithm called Accelerated Nonsmooth Stochastic Gradient Descent (ANSGD), which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that achieves the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons, in which ANSGD significantly outperforms previous subgradient descent algorithms including SGD.