DASH: Data-Aware Scheduling at Higher scale

While computing power of supercomputers keeps on increasing at an exponential rate, their capacity to manage data movement experiences some limits. It is expected that this imbalance will be one of the key limitation to the development of future HPC applications. We propose to rethink how I/O is managed in supercomputers. More specifically, the novelty of this project is to account for known HPC application behaviors (periodicity, limited number of concurrent applications) to define static strategies. We expect that those strategies can be turned into more efficient dynamic strategies than current strategies. In this study, we plan to include a dynamicity provision to cope with any uncertain behavior of applications. We also plan to research how to model and include emerging technologies. Finally we plan to explore the importance and impact of reliabilty and energy-efficiency into I/O management strategies. Main changes with regard to the pre-proposition The current proposition is in the line to the pre-preposition on the research directions considered. There are however some modification made after taking into account the evaluations. Specifically, we have extended the experimental portion of the project in order to increase dissemination and exploitation of our results: • The postdoc position is now an engineer position to be able to develop a library based on the obtained results; • The project is now planned on 48 months instead of 36 so that the library can contain all results obtained during the PhD; • A collaboration has been initiated with Florin Isaila (UC3M) to provide expertise and a framework to integrate and evaluate the results (see Section 2.2, Task 4); • A collaboration has been initiated with an industrial partner, Jean-Thomas Acquaviva (DDN) to integrate and evaluate the use of Burst-Buffers. To accomodate this, we made some minor changes budget-wise: (i) the travel budget has been increased to take into account the collaboration with Florin Isaila; (ii) conference travel costs have also been increased to cover the additional year added to the project. Finally, we plan to organize a workshop dedicated to the problematics around I/O management at the beginning of the third year of the project in order to increase collaboration between the French I/O community (research teams in Rennes, and Grenoble, but also industrial partners) and some European partners (Madrid, Barcelona). Project participants Organization Name Professional Title Contribution research time Role and Responsibilities Inria Bordeaux Sud Ouest Guillaume Aupy CR INRIA 24PM (50%) Project leader: he will ensure that everything goes according to plan. He will cosupervise the PhD student and supervise the interns and engineer Inria Bordeaux Sud Ouest Emmanuel Jeannot DR INRIA 6PM He will help in supervising the PhD student Inria Bordeaux Sud Ouest XXX Research Intern funded by ANR 6PM They will work in defining the problem and models Inria Bordeaux Sud Ouest XXX PhD Student funded by ANR 36PM They will work in the algorithm design and implementaton Inria Bordeaux Sud Ouest XXX Engineer funded by ANR 12PM They will help with the software development and dissemination of the results Inria Bordeaux Sud Ouest XXX Research Intern funded by ANR 6PM They will work in exploring problematics related to reliability and energy-efficiency G. Aupy, DASH (Défi 7 > Axe 7 > Infrastructures de HPC et traitement massif de données) 2 1 Context, position, objectives