Patent ID: 9552481
Date: 2017-01-24
CPC Classifications: G06F

Claim:
1. A computer-implemented method for monitoring programs, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising: placing a program within an enclave to perform a security analysis of the program to ascertain whether the program engages in malicious functionality, the enclave comprising a protected address space that code outside of the protected address space is restricted from accessing except through a special central processing unit enclave entry instruction; hooking an application programming interface call within the program in the enclave to monitor the behavior of the program; inserting the enclave entry instruction into code outside of the protected address space that the program accesses through the hooking of the application programming interface call; and monitoring the behavior of the program by executing the program within the enclave in an attempt to force the program to use the hooked application programming interface call in order to access data outside the enclave, wherein: executing the program comprises attempting, by the program, to bypass at least one instruction at an entry point of the application programming interface call; and bypassing the instruction at the entry point of the application programming interface call further causes the program to bypass an enclave exit instruction and trigger an exception that enables revision of the program to perform further hooking and monitoring as part of the security analysis of the program to ascertain whether the program engages in malicious functionality.