Co-mentors Volodymyr Kindratenko Electrical and Computer Engineering William Gropp Computer Science Social impact Contributing to the advancement of machine learning, which is at the core of many modern approaches to solve real-world problems in the fields ranging from education to healthcare to engineering to core sciences. Project description Deep neural networks are at the core of artificial intelligence, machine learning, computer vision, and other advanced applications across many disciplines. Such networks allow computers to "learn and infer" rather than "compute," which is essential for many problems in which models that describe the data are multi-dimensional, non-linear, and generally are too complex for traditional mathematical techniques.  Many deep learning frameworks have been developed over the course of past decade providing advanced neural network construction, training, and inference functionality. However, vast majority of these codes have been developed for a single compute node execution, which precludes them from training complex network models using large datasets in acceptable time. The challenge is to redesign existing or develop new frameworks that can take advantage of heterogeneous computing platforms to speed up the network training tasks while providing easy to use programming abstractions for domain scientists. In this project, students will analyze open-source deep learning software frameworks and will work on optimizing them and removing bottlenecks in order to improve performance of the applications relying on these codes. Students will be expected to contribute their changes back to these codes, as well as making new codes open sourced. The project will contribute to the development of NSF-funded computer system for deep learning and will result in open-source software that will be deployed on this system. Students will learn about parallel programming systems such as MPI, OpenMP, and OpenCL, how to study their performance, and techniques for improving that performance.