The case for extensible operating systems for exascale

We predict that, initially, system software for exascale systems will be rapidly evolving. New technologies will become available and different ways of using them will come and go. New types of applications and programming models will also introduce new requirements. Features traditionally provided by operating systems (OSes) may not be needed in some cases, while new mechanisms, such as dealing with a much more complex memory hierarchy and the ability to work around faults or operate in degraded mode, need to be added. General OS research will not address exascale-specific challenges since their focus is on consumer OSes and data centers [16]. Also, these OSes are not always suitable for HPC [5], and wont be for exascale. Issues such as OS-noise [24, 15], fault tolerance, and scalability need to be specifically addressed. Furthermore, an exascale OS must enable novel methods of using the hardware in application-specific ways e.g., [6]. Mainstream OS researchers working on parallel systems, often aim their efforts at MapReduce [8, 9] and Hadoop [1], which are only one type of parallel applications and not representative of the range of application types and algorithms expected at exascale. We propose extremely lightweight kernels that can be extended by the runtime system (trusted) or the applications themselves (not trusted). This will result in highly efficient system software tailored to a given architecture, the current usage model, and the application(s) running at the moment. While some of the research necessary to accomplish this has been done in the past, a lot remains to be done to make this a reality. Running untrusted user-level code inside an OS kernel has been studied in the 1990’s but has not really caught on. We believe the time has come to resurrect kernel extensions and combine them with lightweight kernels for OSes that run on highly-parallel systems. Exascale system kernels must be able to quickly adapt to the type of application currently running as well as adapt to the diverse and evolving types of systems. They must do this without sacrificing performance or scalability. Lightweight kernels [34, 20, 27, 22, 23] have proved themselves running highly parallel systems. Due to their smallness, they are relatively easy to expand. However, a generalized method is needed to allow customization for a specific environment and allow applications to tailor kernel behavior at runtime; e.g., to set scheduling memory access policy. While counterintuitive, small sections of interpreted code can provide the performance, flexibility, and protection necessary for kernel extensions [14, 18, 10]. In [11] the authors wonder whether extensible systems lead us astray. Again, their points are mainly directed at general purposes OSes and do not address the need of OSes specific to exascale. In [17] the Exokernel designers make many of the points for kernel extensions we make. Given the specialized environment and usage of a massively parallel system, we need even fewer of the mechanisms an Exokernel provides. Going with the philosophy of removing as much code and mechanism from the kernel as possible, we are proposing a kernel that is even more lightweight than the Exokernel. However, such a kernel needs to be extensible by the runtime system to tailor it to a specific machine and usage environment. Applications also need to be able to extend the OS for their specific needs. The latter needs to be done in a protected manner. The ideas presented in this proposal are explored in more detail in [28].