Reference number : GC-2017-Q1-03424, amendment number : , agreement type : G, recipient legal name : Duvenaud, David (University of Toronto), recipient country : CA, recipient province : nan, recipient city : Ontario, recipient postal code : nan, program name english : Discovery Grants Program - Individual, program name french : Programme de subventions à la découverte - individuelles, program purpose english : The objective of this program is to allow the automatic construction of complex, interpretable models directly from data, automating analysis previously done by experts. 
I will achieve this by further developing tools for automatically building structured deep generative models. 
Such models assert that the data has a hidden underlying structure, observed indirectly through complex, noisy processes. 
This will require novel inference, training and evaluation procedures, but will enable advances in a wide variety of machine learning applications, including automated chemical design, and model-based deep reinforcement learning. 
 
Specifically, we propose to develop a new model class having arbitrarily complex latent variable structure, and deep nonlinear observation models. This research will extend recent lines of inquiry that combine the best aspects of probabilistic graphical models (PGMs) and neural networks. These new models combine the interpretability and tractability of structured graphical models (such as clustering, time-series, or topic models) with the flexibility of neural network-based likelihood functions. This combination can sidestep the rigid modeling assumptions made by PGMs such as Gaussianity. It also sidesteps limitations of neural networks, namely their relative uninterpretability, as well as the requirement of having very large datasets. 
 
Such model classes have been previously proposed, but their usefulness was limited by slow inference methods. We outline a concrete plan for exploiting the structure of the models themselves to develop efficient inference techniques. These methods extend recently-developed variational autoencoders (VAEs) to produce structured variational autoencoders (SVAEs). 
 
These inference methods have the potential to make automatic model-building frameworks practical. In particular, we plan to extend existing matrix decomposition grammars in two ways. First, to handle non-linear observation models connecting different types of latent structures to each other and to the data. Second, to be scalable enough to handle real-world datasets. This modeling framework already contains many standard machine learning and machine vision models as special cases. Providing it with flexible neural-net observation models would mean that it would also contain most deep learning models as special cases, as well as a host of as-yet-undeveloped new model classes. Using fast recognition networks to do inference would mean that inference in this large model class would be scalable., program purpose french : The objective of this program is to allow the automatic construction of complex, interpretable models directly from data, automating analysis previously done by experts. 
I will achieve this by further developing tools for automatically building structured deep generative models. 
Such models assert that the data has a hidden underlying structure, observed indirectly through complex, noisy processes. 
This will require novel inference, training and evaluation procedures, but will enable advances in a wide variety of machine learning applications, including automated chemical design, and model-based deep reinforcement learning. 
 
Specifically, we propose to develop a new model class having arbitrarily complex latent variable structure, and deep nonlinear observation models. This research will extend recent lines of inquiry that combine the best aspects of probabilistic graphical models (PGMs) and neural networks. These new models combine the interpretability and tractability of structured graphical models (such as clustering, time-series, or topic models) with the flexibility of neural network-based likelihood functions. This combination can sidestep the rigid modeling assumptions made by PGMs such as Gaussianity. It also sidesteps limitations of neural networks, namely their relative uninterpretability, as well as the requirement of having very large datasets. 
 
Such model classes have been previously proposed, but their usefulness was limited by slow inference methods. We outline a concrete plan for exploiting the structure of the models themselves to develop efficient inference techniques. These methods extend recently-developed variational autoencoders (VAEs) to produce structured variational autoencoders (SVAEs). 
 
These inference methods have the potential to make automatic model-building frameworks practical. In particular, we plan to extend existing matrix decomposition grammars in two ways. First, to handle non-linear observation models connecting different types of latent structures to each other and to the data. Second, to be scalable enough to handle real-world datasets. This modeling framework already contains many standard machine learning and machine vision models as special cases. Providing it with flexible neural-net observation models would mean that it would also contain most deep learning models as special cases, as well as a host of as-yet-undeveloped new model classes. Using fast recognition networks to do inference would mean that inference in this large model class would be scalable., agreement title english : Efficient Inference in Compositional Generative Models, agreement title french : Efficient Inference in Compositional Generative Models, agreement number : RGPIN, agreement value : 140000.0, agreement start date : 2017-05-10, agreement end date : nan, description english : nan, description french : nan, expected results english : nan, expected results french : nan, additional information english : Grant or Award spanning more than one fiscal year. (2017-2018 to 2022-2023), additional information french : Subvention ou bourse octroyée s'appliquant à plus d'un exercice financier. (2017-2018 à 2022-2023), owner organization : nserc-crsng, owner organization title : Natural Sciences and Engineering Research Council of Canada | Conseil de recherches en sciences naturelles et en génie du Canada