The Quest for Formally Secure Compartmentalizing Compilation

Severe low-level vulnerabilities abound in today's computer systems, allowing cyber-attackers to remotely gain full control. This happens in big part because our programming languages, compilation chains, and architectures too often trade off security for efficiency. The semantics of mainstream low-level languages like C is inherently insecure, and even for safer languages, all guarantees are lost when interacting with low-level code, for instance when using low-level libraries. This habilitation presents my ongoing quest to build formally secure compartmentalizing compilation chains that defend against such attacks. In particular, we propose several formal definitions that characterize what it means for a compartmentalizing compilation chain to be secure, both in the case of safe and of unsafe source languages. We start by investigating what it means for a compilation chain to provide secure interoperability between a safe source language and linked target-level code that is adversarial. In this model, a secure compilation chain ensures that even linked adversarial target-level code cannot break the security properties of a compiled program any more than some linked source-level code could. However, the precise class of security properties one chooses to preserve crucially impacts not only the supported security goals and the strength of the attacker model, but also the kind of protections the compilation chain has to introduce and the kind of proof techniques one can use to make sure that the protections are watertight. We are the first to thoroughly explore a large space of secure compilation criteria based on the preservation against adversarial contexts of various classes of trace properties such as safety, of hyperproperties such as noninterference, and of relational hyperproperties such as trace equivalence. We then extend secure compartmentalizing compilation to unsafe languages like C and C++. We propose a new formal criterion for secure compilation schemes from such unsafe languages, expressing end-to-end security guarantees for software components that may become compromised after encountering undefined behavior---for example, by accessing an array out of bounds. Our criterion is the first to model dynamic compromise in a system of mutually distrustful components with clearly specified privileges. It articulates how each component should be protected from all the others---in particular, from components that have encountered undefined behavior and become compromised. To illustrate this model, we construct a secure compilation chain for a small unsafe language with buffers, procedures, and components, targeting a simple abstract machine with built-in compartmentalization. We give a careful proof (mostly machine-checked in Coq) that this compiler satisfies our secure compilation criterion. We, moreover, show that the protection guarantees offered by the compartmentalized abstract machine can be achieved at the machine-code level using either software fault isolation or a tag-based reference monitor. Finally, we discuss the perspectives of scaling such formally secure compilation to realistic low-level programming languages like C.