Edouard Pauwels: What does back propagation compute?
Channel: Centre International de Rencontres Mathématiques
Date: 2020-04-07

We are interested in nonsmooth analysis of backpropagation as implemented in modern machine learning librairies, such as Tensorflow or Pytorch. First I will illustrate how blind application of
differential calculus to nonsmooth objects can be problematic, requiring a proper mathematical model.
Then I will introduce a weak notion of generalized derivative, named conservativity, and illustrate how it complies with calculus and optimization for well structured objects. We provide stability results for empirical risk minimization similar as in the smooth setting for the combination of nonsmooth automatic differentiation, minibatch stochastic approximation and first order optimization. This is joint work with Jérôme Bolte.

Keywords : optimization, non-smooth analysis, machine learning

MSC Codes :
65K05 - Mathematical programming methods
65K10 - Optimization and variational techniques
68T99 - None of the above but in this section

Recording during the meeting "Optimization for Machine Learning " the March 09, 2020 at the Centre International de Rencontres Mathématiques (Marseille, France)

Filmmaker: Guillaume Hennenfent

Find this video and other talks given by worldwide mathematicians on CIRM's Audiovisual Mathematics Library: http://library.cirm-math.fr. And discover all its functionalities: 
- Chapter markers and keywords to watch the parts of your choice in the video
- Videos enriched with abstracts, bibliographies, Mathematics Subject Classification
- Multi-criteria search by author, title, tags, mathematical area