2 A Log Based Approach To Isolating OS Kernels From Device Driver Defects

The user experience of modern computing systems is greatly enriched by the availability of hardware I/O devices that provide a rich variety of functions including data storage (e.g disk and flash drives), connection to the Internet (e.g network cards, webcams), and entertainment (e.g. speakers, GPUs). I/O devices are quite popular in today’s computing environments (Handheld devices, PCs, Cloud Computing Servers) because they are quite easy to setup for use. In most cases, this involves no more than installing a device driver—a piece of software that is often implemented as an Operating System (OS) kernel module—to enable the OS manage the device. However, defects in device drivers are a major source of OS failures, either by crashing the driver (and consequently the whole system) or by corrupting the kernel or applications. In the Linux kernel, the error rates of drivers are several times that of any other subsystem [12], while up to 85% of Windows XP failures [48, 24] are caused by driver errors. The relatively poor quality of driver code compared to the rest of the kernel could be attributed to a couple of factors. First, most driver developers are not kernel experts, yet they are relied on to write code that will execute correctly in modern OS kernels that rapidly grow in complexity. Second, the large number of drivers (tens of thousands) running at customer sites [32] makes it impractical for OS vendors to test all of them thoroughly. Dynamic binary analysis has been effectively used by dynamic correctness checkers (a.k.a. lifeguards ) for detecting errors in the execution of unmodified application binaries and thus improving the reliability and security of computing systems. Instruction grain lifeguards can detect the most subtle program faults including low level issues such as memory [34] and security [16, 35] errors, as well as high level ones such as concurrency errors [46, 53, 23] and interface errors in multilingual programs [30]. Lifeguards are deployed using either instrumentation or logging of the monitored application. Dynamic binary instrumentation (DBI) approaches inject lifeguard code into the application using software [7, 10, 34], or hardware [15], while logging based approaches capture an instruction grained execution trace of the application using software [13, 36] or hardware [9] for online lifeguard analysis. Dynamic binary analyses seem to be well suited for use on production drivers which are often distributed in binary form for both convenience and proprietary reasons. This thesis work explores the effectiveness of using dynamic binary analysis for online detection of production driver faults, both of the low level nature such as buffer overflows, as well as high level ones like OS protocol violations. However, interrupts and preemption in modern OSes running on chip multiprocessor (CMP) systems pose a major challenge to the precision of instrumentation based instruction grained lifeguards since the atomic execution of an application instruction and the corresponding lifeguard code is no longer guaranteed. Signal handlers pose a similar problem for instruction grained application lifeguards. Current solutions for enforcing atomic execution of application and lifeguard code using transactional memory [14], or disabling preemption [34, 33], are not applicable for monitoring drivers, and kernel mode execution in general, since they conflict with OS preemptibility. On the other hand, this is not an issue for log based approaches which can leverage solutions for obtaining totally ordered multi-threaded execution traces on CMPs [40, 51, 26] to order lifeguard actions. However, log based approaches require error containment mechanisms due to their delayed correctness checking. My thesis is that log based lifeguards can use dynamic binary analysis to detect defects in unmodified device driver binaries, and therefore be used for isolating applications and OS kernels from defective drivers. In this thesis, driver faults are assumed to be developer errors and not malicious attacks, in other words the proposed lifeguards can be used to protect the OS kernel from defective drivers but not from malicious ones. The rest of this