text: 352,26,1243551404297342982,888216099757490176,Maithra Raghu,"A Survey of Deep Learning for Scientific Discovery To help facilitate using DL in science, we survey a broad range of deep learning methods, new research results, implementation tips & many links to code/tutorials Paper <LINK> Work with @ericschmidt  Thread⬇️ <LINK> We begin with some high level considerations: (i) template ways in which deep learning can be used in scientific problems (ii) overviews of the entire end-to-end deep learning design process (iii) highlights of (when to use) key alternate machine learning methods We provide links to incredible resources developed by the community: software packages & high level APIs, freely available DL tutorials, sites with summaries/discussions/code of new research, repositories of DL pipelines & pretrained models, data curation & analysis packages <LINK> We then describe core models/tasks/methods, including CNNs (detection, segmentation, registration, many others), graph NNs, sequence models + tasks (RNNs, transformers, attention, embeddings, Q&A, seq2seq, etc) with links to science use cases, tutorials and code throughout <LINK> We overview other powerful methods for training neural networks, such as transfer learning, domain adaptation, multitask learning and weak supervision. <LINK> In many (scientific) use cases not much data may be available to train machine learning models. Requiring less (labelled) data is a very active research area, and we overview new advances in (i) self-supervision (ii) semi-supervised learning (iii) data augmentation (iv)denoising <LINK> Central to many scientific problems is going from *predictions* to *understanding*: identifying underlying mechanisms & key data features. We survey results in interpretability & representation analysis enabling data feature attribution & insights on model hidden representations <LINK> We also (i) highlight core ideas and possible use cases of deep generative models and deep reinforcement learning (ii) provide implementation tips for getting started (explore data, try simple methods, start with known models/algorithms) and for debugging/improving performance The best part of writing this survey was learning even more about the incredible work being done by the community across research, teaching courses, developing/opensourcing code, in-depth tutorials. Was very hard to reference it all! We welcome pointers to other related work!",https://arxiv.org/abs/2003.11755,"Over the past few years, we have seen fundamental breakthroughs in core problems in machine learning, largely driven by advances in deep neural networks. At the same time, the amount of data collected in a wide array of scientific domains is dramatically increasing in both size and complexity. Taken together, this suggests many exciting opportunities for deep learning applications in scientific settings. But a significant challenge to this is simply knowing where to start. The sheer breadth and diversity of different deep learning techniques makes it difficult to determine what scientific problems might be most amenable to these methods, or which specific combination of methods might offer the most promising first approach. In this survey, we focus on addressing this central issue, providing an overview of many widely used deep learning models, spanning visual, sequential and graph structured data, associated tasks and different training methods, along with techniques to use deep learning with less data and better interpret these complex models --- two central considerations for many scientific use cases. We also include overviews of the full design process, implementation tips, and links to a plethora of tutorials, research summaries and open-sourced deep learning pipelines and pretrained models, developed by the community. We hope that this survey will help accelerate the use of deep learning across different scientific domains. ",A Survey of Deep Learning for Scientific Discovery,9,"['A Survey of Deep Learning for Scientific Discovery\n\nTo help facilitate using DL in science, we survey a broad range of deep learning methods, new research results, implementation tips &amp; many links to code/tutorials\n\nPaper <LINK>\n\nWork with @ericschmidt\n \nThread⬇️ <LINK>', 'We begin with some high level considerations: (i) template ways in which deep learning can be used in scientific problems (ii) overviews of the entire end-to-end deep learning design process (iii) highlights of (when to use) key alternate machine learning methods', 'We provide links to incredible resources developed by the community: software packages &amp; high level APIs, freely available DL tutorials, sites with summaries/discussions/code of new research, repositories of DL pipelines &amp; pretrained models, data curation &amp; analysis packages https://t.co/JloKcxtAO4', 'We then describe core models/tasks/methods, including CNNs (detection, segmentation, registration, many others), graph NNs, sequence models + tasks (RNNs, transformers, attention, embeddings, Q&amp;A, seq2seq, etc) with links to science use cases, tutorials and code throughout https://t.co/ADfMsZ0W0k', 'We overview other powerful methods for training neural networks, such as transfer learning, domain adaptation, multitask learning and weak supervision. https://t.co/QPdWPCOjgS', 'In many (scientific) use cases not much data may be available to train machine learning models. Requiring less (labelled) data is a very active research area, and we overview new advances in (i) self-supervision (ii) semi-supervised learning (iii) data augmentation (iv)denoising https://t.co/Ag028UbRI9', 'Central to many scientific problems is going from *predictions* to *understanding*: identifying underlying mechanisms &amp; key data features. We survey results in interpretability &amp; representation analysis enabling data feature attribution &amp; insights on model hidden representations https://t.co/oCIkJaqgE2', 'We also (i) highlight core ideas and possible use cases of deep generative models and deep reinforcement learning (ii) provide implementation tips for getting started (explore data, try simple methods, start with known models/algorithms) and for debugging/improving performance', 'The best part of writing this survey was learning even more about the incredible work being done by the community across research, teaching courses, developing/opensourcing code, in-depth tutorials. Was very hard to reference it all! We welcome pointers to other related work!']",20,03,2382