Architectural support for software security

Program execution can be tampered by attackers through exploitation of diverse sequential and concurrent errors. Many software and hardware approaches have been presented to validate data integrity and check program execution. Software approaches suffer large performance degradation that limits their application on run-time systems. Hardware approaches reduce the performance penalty, but current solutions incur high logic and storage overhead. To address these challenges, we propose to augment multiple system layers, i.e., hardware architecture, compiler, and operating system, in a coordinated manner to provide efficient monitoring with the same or better level of protection than current solutions. Our approaches consider security during the design process, along with other metrics such as cost, performance, and power so as to: 1) augment the attack detection rate in existing control-flow monitors, 2) reduce the overhead for doing dynamic information flow tracking in a single core architecture, 3) elevate the efficacy for doing dynamic information flow tracking in the multi-core scenario, 4) improve the efficacy and the efficiency of multithread data isolation. In the first case, we found that the current control-flow monitor systems are still vulnerable to non-control data attacks. A malicious user can deceive the detection mechanism that the attack actions are part of the normal behavior. It is possible because of the inherent limitations in the information used to validate the execution profile at run-time, the execution signature. We proposed to leverage the existing branch prediction mechanism, including the branch target buffer, to monitor the execution program. We incorporated information like the binary branch taking history and the expected (ahead) path into the program’s signature in order to make it stronger. We tested our approach under conditions that current approaches fail to detect non-control data attacks. Our solution can detect the traditional control-flow attacks as well as the non-control data attacks, reduce the implementation complexity, and get a low execution overhead. Dynamic information flow tracking (DIFT) is a popular method to trace input propagation in a computer system so as to prevent memory corruptions. However, DIFT slows down the program execution and many implementaions require large memory space for tag storage (also known as metadata). In order to alleviate these problems, we proposed compile-assisted aggregation of program variables according to their taints and memory page allocation. We also proposed modifications in the memory management unit and operating system for in-core run-time tracking. Our approach almost eliminates the storage overhead and significantly reduces the performance degradation caused by dynamic tracking and processing. Furthmore, when adopting dynamic information flow tracking in multi-core architectures, we found that inconsistency issues between data and metadata processing may arise because data and metadata are proccessed in different places (cores) at different time. Current solutions enforce the same data order for metadata processing, and do not allow independent metadata to be accessed out-of-order. As a result, the execution overhead is large. We proposed a centralized architectural approach that uses the existing cache coherence unit to enforce the correct order for metadata when it is needed and meanwhile allow out-of-order metadata processing. Our results show a considerable overhead reduction compared to in-order processing, and significant improvement over the prior distributed approach in terms of implementation complexity and execution overhead. Finally, ensuring data integrity and confidentiality in multi-thread applications is hard to accomplish because all the threads are running in the same application space. Software mechanisms for thread isolation can prevent an unauthorized thread from accessing other threads’ private data. However, the cost of in-lining checking slows down the application execution. We proposed to reduce the code complexity by moving the checking mechanism from the software domain to the hardware domain. We showed that with system library modification and operating system support, a hardware monitoring mechanism can provide higher security more efficiently. In summary, our proposed enhancements to hardware architecture, compiler, and operating system contribute to the state of the art in the following ways: • Elevating the security defense level of control-flow monitors by making the program signature stronger. • Reducing the storage overhead and program execution slow-down of DIFT by aggregating trusted and untrusted data in different pages. • Allowing secure out-of-order metadata processing on multi-core DIFT and enforcing metadata access order only when data dependencies are present. • Ensuring data integrity and confidentiality in multi-thread programs by moving the monitoring mechanism from the software domain to the hardware domain.