Patent ID: 8555386

Claim:
An automated computer-implemented method for investigating a presence of malicious code in an application program stored on a subject computer system, the subject computer system including a processor, memory, and an operating system that facilitates user-mode data processing and kernel-mode data processing wherein a plurality of system application program interfaces (APIs) provide access to user-mode and kernel-mode system functions, the method comprising: emulating instructions of the application program according to a first emulation mode wherein the instructions of the application program are carried out sequentially in a first virtual execution environment that represents the user-mode data processing of the operating system; detecting, among the instructions of the application program, a system API call requesting execution of a user-mode system function; in response to detecting the system API call, emulating instructions of the user-mode system function called by the API call according to a second emulation mode wherein the instructions of the user-mode system function are carried out sequentially in a second virtual execution environment that represents the user-mode data processing of the operating system, including tracking certain processor and memory states affected by the instructions of the user-mode system function; and analyzing results of the emulating of the application program instructions according to the first emulation mode for any presence of malicious code.