Andrew G. Wilson - How do we build models that learn and generalize?
Channel: Secondmind
Date: 2021-04-16

Abstract: To answer scientific questions, and reason about data, we must build models and perform inference within those models. But how should we approach model construction and inference to make the most successful predictions? How do we represent uncertainty and prior knowledge? How flexible should our models be? Should we use a single model, or multiple different models? Should we follow a different procedure depending on how much data are available? How do we learn desirable constraints, such as rotation, translation, or reflection symmetries, when they don't improve standard training loss? In this talk I will present a philosophy for model construction, grounded in probability theory. I will exemplify this approach with methods that exploit loss surface geometry for scalable and practical Bayesian deep learning, and resolutions to seemingly mysterious generalization behaviour such as double descent. I will also consider prior specification, generalized Bayesian inference, and automatic symmetry learning. 

Preprints: The talk is primarily based on https://arxiv.org/abs/2002.08791, and it also touches on https://arxiv.org/abs/2002.12880 and https://arxiv.org/abs/2010.11882.

Speaker: Andrew G. Wilson is Assistant Professor at Courant Institute of Mathematical Sciences and Center for Data Science, New York University. More info can be found at the personal website: https://cims.nyu.edu/~andrewgw/

This talk was given at Secondmind Labs, as a part of our (virtual) research seminar. Our research seminar is where we exchange ideas with guest speakers, keeping you up to date with the latest developments and inspiring research topics. Occasionally, Secondmind researchers present their own work as well. You can find a complete list of speakers at https://www.secondmind.ai/labs/seminars/. Learn more about Secondmind Labs at https://www.secondmind.ai/labs/​