Reference number : GC-2017-Q1-02448, amendment number : , agreement type : G, recipient legal name : Trappenberg, Thomas (Dalhousie University), recipient country : CA, recipient province : nan, recipient city : Nova Scotia|Nouvelle-Écosse, recipient postal code : nan, program name english : Discovery Grants Program - Individual, program name french : Programme de subventions à la découverte - individuelles, program purpose english : Deep neural networks is a machine learning technique based on a model of neuron-like elements. The "deep" refers here to networks with many layers that are capable of representing complex functions. Such models have been proven to be extremely valuable as they can learn hierarchical data representations from examples and predict previously unseen data. Such techniques enabled much of the recent progress in computer vision and pattern recognition, and the advancements of deep neural networks was made possible by the availability of large data sets (Big Data), some refinement of algorithms, and the availability of increased computational power with specialized processors such as graphical processing units (GPUs). Companies like Google, Microsoft, Amazon and Facebook, and also numerous smaller engineering firms and market analysis firms are embracing this new technology. 
 
However, many applications do not have the luxury of having a large set of examples available to train complex models. In the research program proposed here I am trying to develop methods for applications with limited data, which includes a variety of situations such as small number of measurements, biased examples, or examples from peripheral data. Applications with limited data are common. For example, in the ocean science community it is common to have a fairly small coverage of measurements compared to the vastness of the oceans. My lab has started to investigate how artificial data can augment pre-training and facilitate the narrowing of the search space. For example, there exist a variety of physical models of ocean dynamics such as the exchange of CO2 with the atmosphere. While these models are generally considered to be insufficient for predictions on the required scale, it is possible that these simulations can provide data for the pre-training of deep networks. 
 
Another area that promises a whole new level of techniques to training deep networks is the guidance of the learning process by domain experts, also called transfer learning. Domain experts include hereby either deep neural networks that have been trained on complementary or similar domains, or even human experts. While transfer learning has been considered for some time, the use of deep networks to learn appropriate communication channels for learning opened exciting new possibilities for much progress in this area. Such expert training would provide network training that parallels human learning. Such techniques could further drastically reduce the need of large training sets. 
 
The long-term goal of this research program is to develop specific methods and tools to both evaluate a machine-learning problem in terms of its complexity and data need, and to provide a comprehensive toolbox to apply pre-learning and expert transfer learning strategies to problem domains that suffer from limited data., program purpose french : Deep neural networks is a machine learning technique based on a model of neuron-like elements. The "deep" refers here to networks with many layers that are capable of representing complex functions. Such models have been proven to be extremely valuable as they can learn hierarchical data representations from examples and predict previously unseen data. Such techniques enabled much of the recent progress in computer vision and pattern recognition, and the advancements of deep neural networks was made possible by the availability of large data sets (Big Data), some refinement of algorithms, and the availability of increased computational power with specialized processors such as graphical processing units (GPUs). Companies like Google, Microsoft, Amazon and Facebook, and also numerous smaller engineering firms and market analysis firms are embracing this new technology. 
 
However, many applications do not have the luxury of having a large set of examples available to train complex models. In the research program proposed here I am trying to develop methods for applications with limited data, which includes a variety of situations such as small number of measurements, biased examples, or examples from peripheral data. Applications with limited data are common. For example, in the ocean science community it is common to have a fairly small coverage of measurements compared to the vastness of the oceans. My lab has started to investigate how artificial data can augment pre-training and facilitate the narrowing of the search space. For example, there exist a variety of physical models of ocean dynamics such as the exchange of CO2 with the atmosphere. While these models are generally considered to be insufficient for predictions on the required scale, it is possible that these simulations can provide data for the pre-training of deep networks. 
 
Another area that promises a whole new level of techniques to training deep networks is the guidance of the learning process by domain experts, also called transfer learning. Domain experts include hereby either deep neural networks that have been trained on complementary or similar domains, or even human experts. While transfer learning has been considered for some time, the use of deep networks to learn appropriate communication channels for learning opened exciting new possibilities for much progress in this area. Such expert training would provide network training that parallels human learning. Such techniques could further drastically reduce the need of large training sets. 
 
The long-term goal of this research program is to develop specific methods and tools to both evaluate a machine-learning problem in terms of its complexity and data need, and to provide a comprehensive toolbox to apply pre-learning and expert transfer learning strategies to problem domains that suffer from limited data., agreement title english : Deep learning with limited data, agreement title french : Deep learning with limited data, agreement number : RGPIN, agreement value : 130000.0, agreement start date : 2017-05-10, agreement end date : nan, description english : nan, description french : nan, expected results english : nan, expected results french : nan, additional information english : Grant or Award spanning more than one fiscal year. (2017-2018 to 2022-2023), additional information french : Subvention ou bourse octroyée s'appliquant à plus d'un exercice financier. (2017-2018 à 2022-2023), owner organization : nserc-crsng, owner organization title : Natural Sciences and Engineering Research Council of Canada | Conseil de recherches en sciences naturelles et en génie du Canada