Patent Description:
<CIT> describes a supervisory process which controls a set of security applications to perform offline scanning and repair of multiple virtual machines (VMs) running on a host system. If a problem is detected during the offline scanning, the VM may be notified of the problem or instructed to roll back to a last saved state before the problem occurred.

<CIT> and <CIT> describe a method for analyzing an application in a runtime environment involving applying stimuli to the application to cause the application to reach a "region of interest". The region of interest is monitored to determine whether it corresponds to improperly behaving code.

<CIT> describes a system for detecting agent presence for self-healing. An out-of-band monitoring process and reports crashed processes to an enterprise security controller which may instruct affected machines to take remedial action.

<CIT> describes a program for detecting malware based upon behavior of a system. An expected activity level of resources of the system is calculated and compared to an actual determined level of activity. If a deviation is detected between the expected and actual determined level of activity a source of unexpected activity is determined and, if not legitimate, is classified as malware.

The present application may be more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings.

Systems, methods, and equivalents associated with process verification are described. As discussed above, there are many techniques for providing device security, While an antivirus application may be a frequently used technique, often, malicious attacks begin by attempting to shut down or otherwise impede the antivirus application itself. To protect the antivirus application, a protection process may be embedded in an environment that is isolated from a general environment operating the antivirus. While the antivirus itself could theoretically operate from the isolated environment, this may make it more difficult to update the antivirus against future threats, as well as potentially increasing the costs associated with delivering the isolated environment with the device. This is because the more functionality that is added to the isolated environment, the more expensive it becomes to maintain and protect the isolated environment. Additionally, it may be difficult to interpret the semantics of data structures and memory layouts from external to the file system. Consequently, processes operating from the isolated environment may be designed to be lightweight, secure, and use limited resources.

As disclosed herein, a protection process embedded in an isolated environment may be configured to prevent malicious interference with processes operating on a device's general operating environment (or other operating environments of the device). While the processes protected may be security related (e.g., antivirus applications, firewall applications, digital rights management applications), other processes may also be protected (e.g., operating systems, productivity applications, games, and so forth). This protection may be achieved by the protection process directly manipulating the data (e.g., executable instructions) used by the protected process as the protected process is operating so as to change the behavior of the protected process during operation of the protected process. The protection process may then verify that the behavior change has occurred, and trigger a repair of the protected process in the event that the behavior change is not exhibited.

It is appreciated that, in the following description, numerous specific details are set forth to provide a thorough understanding of the examples. However, it is appreciated that the examples may be practiced without limitation to these specific details. In other instances, methods and structures may not be described in detail to avoid unnecessarily obscuring the description of the examples. Also, the examples may be used in combination with each other.

"Module", as used herein, includes but is not limited to hardware, firmware, software stored on a computer-readable medium or in execution on a machine, and/or combinations of each to perform a function(s) or an action(s), and/or to cause a function or action from another module, method, and/or system. A module may include a software controlled microprocessor, a discrete module, an analog circuit, a digital circuit, a programmed module device, a memory device containing instructions, and so on. Modules may include gates, combinations of gates, or other circuit components. Where multiple logical modules are described, it may be possible to incorporate the multiple logical modules into one physical module. Similarly, where a single logical module is described, it may be possible to distribute that single logical module between multiple physical modules.

<FIG> illustrates an example system <NUM> associated with process verification. System <NUM> includes a process <NUM>. Process <NUM> may operate within a general operating environment of system <NUM>. As used herein a general operating environment may be an environment of a system or device that performs general purpose processing functions for the system or device. Thus the general operating environment may provide processing (e.g., by a central processing unit), memory (e.g., by a random access memory), and so forth for an operating system of the device or system, user interfacing applications of the system or device, and so forth. In various examples process <NUM> may perform a security function for system <NUM>, for a device connected to system <NUM>, for a network connected to system <NUM>, and so forth. Thus, process <NUM> may be, for example, an antivirus process, a firewall process, an authentication process, a cryptography process, an intrusion prevention process, a digital rights management process, an intrusion detection process, and so forth. In other examples, process <NUM> may be another type of process warranting protection not primarily related to the security of system <NUM> or another device to which system <NUM> is connected. By way of illustration, process <NUM> may be, for example, an operating system, a game, a network service, and so forth.

<FIG> also includes an isolated environment <NUM>. Isolated environment <NUM> may be an environment of a system that is separate from a general operating environment of that system. This separation may restrict access to data and/or processes stored on or operating from the isolated environment. Thus, certain access privileges may be needed to access data on isolated environment <NUM>, to transmit data to isolated environment <NUM>, and so forth. In other examples, processes operating from isolated environment <NUM> may be able to affect the general operating environment of system <NUM>, while processes from the general operating environment may be restricted from affecting isolated environment <NUM>. Isolated environment <NUM> may be, for example, a TrustZone system on chip, a hypervisor, a system management mode module, an embedded controller, and so forth.

Isolated environment <NUM> includes a protection module <NUM>. Protection module <NUM> may perform various tasks associated with ensuring the continued, proper operation of process <NUM>. To achieve this, protection module <NUM> may modify the behavior of process <NUM> by modifying data associated with process <NUM> while process <NUM> is in operation. Modifying the behavior of process <NUM> may apply a variety of techniques including several different techniques, and may target different portions of the data associated with process <NUM> during different tests, so as to facilitate verification that various portions of process <NUM> are operating properly. The protection module <NUM> alters process <NUM> by altering executable instructions of process <NUM> in memory. By way of illustration, executable instructions may be modified to adjust a target of a memory access or write, to change a value manipulated by process <NUM>. In another example, protection module <NUM> may alter a Boolean value in memory to trigger whether a function of process <NUM> operates. In another example, protection module <NUM> may overwrite null operation instructions with replacement instructions in the executable instructions associated with process <NUM>. The replacement instructions may, for example, call specific functions of process <NUM> that may not otherwise be called. In some examples, the null operation instructions may have been inserted in process <NUM> at compile time by a specially configured compiler. In another example, protection module <NUM> may overwrite a function call in the executable instructions of process <NUM> to call an alternative function call. In another example, protection module <NUM> may alter a function pointer in memory to cause process <NUM> to call an alternative function. Changing which functions are called may change the behavior of process <NUM> in observable ways when, for examples, the functions affect predetermined portions of memory, the functions transmit signals to other components of system <NUM> (e.g., protection module <NUM>), and so forth.

Protection module <NUM> may also verify whether the behavior of process <NUM> has changed in accordance with the modification. Verifying the behavior of the process may also take many forms. For example, protection value <NUM> may verify a value received from process <NUM>. The value may be generated based on the behavior modified by protection module <NUM> and transmitted to protection module <NUM> by process <NUM>. In another example, protection module <NUM> may verify that security reports provided by process <NUM> to protection module <NUM> include data collected as a result of the behavior modification. In this example, protection module <NUM> may control process <NUM> to provide a specific piece of data at a specific memory location or directly to protection module <NUM>, and verify that the specific piece of data is in an expected format. In another example, protection module <NUM> may verify a state of an in memory value modified by process <NUM> during the operation of process <NUM>. The state sought by protection module <NUM> may depend on how protection module <NUM> modified process <NUM>. For example, protection module <NUM> may add function calls to the executable instructions of process <NUM> that cause process <NUM> increment an in memory value a certain number of times. Protection module <NUM> may then verify that the in memory value exhibits an expected value after a certain period of time, after an execution point in process <NUM> has been reached, and so forth.

Upon determining that process <NUM> has been compromised, protection module <NUM> may take a remedial action. The remedial action may be, for example, alerting an entity (e.g., a user, a process capable of restoring process <NUM> to a prior state, a company invested in the security of process <NUM>) that process <NUM> has been compromised, disabling a function of system <NUM> (e.g., access to restricted data and/or processes on system <NUM>), restoring process <NUM> to a known valid state (e.g., by retrieving a backup of process <NUM> from a known secure location and overwriting process <NUM> in a memory), turning off system <NUM>, and so forth. A combination of these functionalities may also be appropriate.

In some examples, protection module <NUM> may operate in connection with a remote device such as a remote server (not shown). This may, for example, provide an additional layer of security for system <NUM> by preventing isolated environment <NUM> and/or protection module <NUM> from being themselves compromised by a malicious entity. Thus, protection module <NUM> may receive instructions for modifying the behavior of process <NUM> from the remote device. In this example, protection module <NUM> may report results of verifying process <NUM> to the remote device. If the remote device determines from the reported results that process <NUM> and/or protection module <NUM> have been compromised, then the remote device may be able to initiate a remedial action (e.g., restoring a component of system <NUM> to a prior state, alerting an administrator).

<FIG> illustrates an example method <NUM>. Method <NUM> may be embodied on a non-transitory processor-readable medium storing processor-executable instructions. The instructions, when executed by a processor, may cause the processor to perform method <NUM>. In other examples, method <NUM> may exist within logic gates and/or RAM of an application specific integrated circuit (ASIC).

Method <NUM> may perform various tasks associated with process verification. Method <NUM> includes modifying data associated with a security process at <NUM>. The data may be modified to cause a behavior change in the security process. This data modification may be performed from a first environment of a device. The first environment may be, for example, an isolated environment. Thus, the isolated environment may be a TrustZone system on chip, a hypervisor, a system management mode module, and an embedded controller. The data modified may reside on a second environment of the device. The second environment may be a primary processing environment of the device. In some examples, the data associated with the security module may be modified by altering in memory executable instructions of the security process.

Method <NUM> also includes verifying proper operation of the security process at <NUM>. The proper operation of the security process may be verified based on the behavior change. By way of illustration, proper operation of the security process may be verified by verifying a value generated based on the changed behavior and received from the security process. In another example, security reports provided by the security process may be verified to include data collected as a result of the changed behavior. In another example, a state of an in memory value may be verified to determine if the in memory value was modified by the security process during the operation of the security process.

Method <NUM> also includes performing a remedial action at <NUM>. The remedial action may be performed when the security process fails to exhibit the behavior change that was sought to be triggered at action <NUM>. The remedial action may be, for example, alerting an entity that the security process may be compromised, restoring the security process to a prior state, deactivating a functionality of a device operating the security process, and so forth.

<FIG> illustrates a method <NUM>. Method <NUM> includes altering executable instructions associated with a security process at <NUM>. The executable instructions may be altered from a protected environment of a device. The security process may be in execution by a general operating environment of the device. Altering the executable instructions may trigger a behavior change in the security process. Method <NUM> may also include verifying whether the security process exhibits the behavior alteration at <NUM>. Method <NUM> also includes restoring the security process to a known valid state at <NUM>. The security process may be restored to the valid state when the security process fails to exhibit the behavior alteration.

<FIG> illustrates a method <NUM> associated with process verification, Method <NUM> includes several actions similar to those described above with reference to method <NUM>. For example, method <NUM> includes altering executable instructions associated with a security process at <NUM>, verifying at <NUM> whether the security process exhibits a behavior alteration caused by altering the executable instructions, and restoring the security process to a known valid state at <NUM>.

Method <NUM> also includes receiving directions from a remote device at <NUM>. The directions may describe how to modify the behavior of the security process at action <NUM>. Method <NUM> also includes reporting results of the verification to the remote device at <NUM>. Method <NUM> also includes receiving a signal directing the restoration of the security process from the remote device at <NUM>.

<FIG> illustrates an example computing device in which example systems and methods, and equivalents, may operate. The example computing device may be a computer <NUM> that includes a processor <NUM> and a memory <NUM> connected by a bus <NUM>. Computer <NUM> includes a process verification module <NUM>. Process verification module <NUM> may perform, alone or in combination, various functions described above with reference to the example systems, methods, and so forth. In different examples, process verification module <NUM> may be implemented as a non-transitory computer-readable medium storing processor-executable instructions, in hardware, software, firmware, an application specific integrated circuit, and/or combinations thereof.

The instructions may also be presented to computer <NUM> as data <NUM> and/or process <NUM> that are temporarily stored in memory <NUM> and then executed by processor <NUM>. The processor <NUM> may be a variety of processors including dual microprocessor and other multi-processor architectures. Memory <NUM> may include non-volatile memory (e.g., read-only memory, flash memory, memristor) and/or volatile memory (e.g., random access memory). Memory <NUM> may also be, for example, a magnetic disk drive, a solid state disk drive, a floppy disk drive, a tape drive, a flash memory card, an optical disk, and so on. Thus, memory <NUM> may store process <NUM> and/or data <NUM>. Computer <NUM> may also be associated with other devices including other computers, devices, peripherals, and so forth in numerous configurations (not shown).

Claim 1:
A system (<NUM>), comprising:
a process (<NUM>) operating in a general operating environment of the system; and
an isolated environment (<NUM>) comprising:
a protection module (<NUM>) to:
modify the behavior of the process (<NUM>) by modifying data associated with the process (<NUM>) while the process (<NUM>) is in operation to cause a behavior change in the process (<NUM>);
verify whether the process (<NUM>) has changed in accordance with the modification; and
take a remedial action when the process (<NUM>) fails to exhibit the behavior change;
wherein the protection module modifies the data associated with the process (<NUM>) by altering in memory executable instructions of the process (<NUM>); and
wherein the executable instructions are modified by at least one of:
altering a Boolean value in memory to trigger whether a function of the process (<NUM>) executes;
overwriting null operation instructions in the executable instructions associated with the process (<NUM>) with instructions that call specific functions, where the null operation instructions were inserted in the process (<NUM>) at compile time by a specially configured compiler;
overwriting a function call in the executable instructions to trigger an alternative function call; and
altering a function pointer in memory.