MAD Ensembles Are Saner Than You Think

environment (as per traditional theorem proving systems); if the check fails at this highest level of abstraction, there is probably no need to start discussing transport protocols, specific cryptographic primitives, or processor power levels. Translation from the global specification to more concrete, lowerlevel specifications can be done automatically, much like compilation for typical programs or planning for database queries, perhaps at multiple intermediate granularities. For example, it may be helpful to partition the global abstract computer to a set of administrativedomain-specific components, since under most scenarios, what hap pens within an administrative domain is not only customized to fit local needs, but may also be private. The breakdown of the highlevel functionality and required properties into domain-local functionality and properties also allows a relaxation of the threat model from a full-blown MAD environment across domains to an honestoperator-faulty-software model within a single domain. Knowledge of particular domain interrelationships such as degrees of trust can inform this refinement on a case-by-case basis. Similarly, the translation can move to smaller subgroups, such as network locality groups to optimize communication patterns, single-rack groups to optimize heat dissipation, and single processor cores to take advantage of shared processing structures. At the bottommost level, the ensemble’s individual elements are programmed, each in its own machine language. Along the translator, an optimizer can be invaluable in this process, both during “compile time” and during runtime tuning. Whereas translation occursvertically through refinement, optimization can operatehorizontally, collapsing redundancy, rearranging computations to improve communication patterns, changing implementations of packaged functionalities (e.g., from a proactive to a reactive routing protocol) to match impedance with other choices (e.g., reactive routing with lazy consistency enforcement). Finally, the execution engine can take the final runtime, verify any proofs of correctness supplied by the translator and optimizer, and execute the system, ensuring that adaptation is triggered as conditions change. The need for explicit, portable assurances, especially within the context of MAD ensembles, is unavoidable and challenging through all stages of the process. At the specification stage, the language must be expressive enough to describe access and flow control restrictions (e.g., “tax IDs should not flow through ISP X’s wires in the clear”), and priorities among different optimization axes (e.g., “use as little memory as possible and then protect from information leak to the extent achievable”). At the optimization stage, it must be possible to keep track of how each program transformation – partitioning, replication, or reordering – affects the constraints of the specification (e.g., split a private, two-party computation without leaking information across the parties). At the execution stage, the system must enforce the semantics of the specification, e.g., ensuring that private information is not placed in the clear on shared memory; it must also make it possible to debug, so that a programmer can figure out what is going on without having to compromise any guarantees made by the system (e.g., restart a deadlocked, privacy-preserving computation without having to compromise the privacy of inputs or partial outputs). Deep down, this vision is founded on a simple (repeatedly rediscovered) principle: specifications should be layered but runtimes need not be. Structure among functionalities helps human designers separate their concerns in relevant units; however, the realities of performance and fault-tolerance optimizations break layer boundaries invalidating the benefits of abstraction. Perhaps it is time to reconsider having the cake (of layered specifications) and eating it too (by producing safe yet optimized, layerless runtimes). 3. MAD DECLARATIVE ENSEMBLES We have been pursuing this agenda in the context of the P2 Declarative Networking Project [1]. In P2, a distributed application is expressed declaratively in OverLog [5], a logic language based on Datalog. The whole ensemble is treated as a distributed database, its algorithm expressed as a continuous query over all elements’ changing states. A compiler translates the logic into event-condition-action rules performed on single elements, optimizes them to minimize data movement (a la System R’s optimizations [8] with network costs included), and plans the resulting runtime as a software dataflow graph, similar to Click [4]. Execution interprets the dataflow graph. Our current research in the P2 project pursues the goals of the MAD ensemble programming agenda, along multiple paths. First, on the language front, we are growing P2’s declarative specification language (OverLog) to include primitives about authentication (e.g., Binder’s “says” construct [2]), information flow (such as d ecentralized labels [6]), and consistency requirements (such as linearizability or session guarantees). This allows OverLog’s expressivity to capture not only algorithmic specifications but also the security and fault-tolerance requirements of the specified application. Furthermore, we are strengthening OverLog’s type system to make the language type safe. Second, we are evolving the P2 compiler and optimizer so that, in addition to dealing with the new language features above, they can provide assurances on the correctness of output runtimes. We are exploring proof-carrying code [7] as an approach to allow programs from potentially untrusted compilers (especially during runtime adaptation) to be shared across domains. P2’s logic-based specification language enables a straightforward connection to a theorem prover for the generation of proofs that go beyond simple memory safety to global properties such as resource quota compliance. We are also adapting the techniques of secure program partitioning [9] to our compiler’s existing (insecure) program partitioning transformations, taking advantage of any information flow annotations. The compiler also encapsulates parts of a program’s specification in known fault-tolerant constructs such as Byzantinefault tolerant replicated state machines to ensure strict consistency guarantees for sensitive parts of a specification. Third, we are migrating the remaining imperative components of the runtime (as well as the debugging and adaptation modules) to a type-safe system language (we are currently considering OCaml); large portions of the runtime adaptation module as well as the compiler and optimizer are themselves written in OverLog, simplifying this process. The goal is to eventually make the whole P2 software stack certifiably memory-safe. Though exciting and promising, these early steps do not make the scope of the MAD ensembles agenda any less ambitious (or mad). Our path is not dictated by feasibility but, rather, by necessity. As cookbook solutions to specific problems for untrusted environments mature, thanks in the largest part to the efforts of the distributed systems and dependability communities, we hope our findings will help to give programmers the tools they need to design, program, verify, and debug large, complex, heterogeneou s deployments across administrative boundaries.