Security policy deployment and enforcement system for the detection and control of polymorphic and targeted malware

The present system and method pertain to the detection of malicious software and processes such as malware. A cloud security policy system receives hashes and behavioral information about applications and/or process executing on user devices. The cloud security policy system records this information and then evaluates the trustworthiness of the hashes based on the information received from the user devices to provide a security policy for the applications and/or processes. The security policy is sent from the cloud security policy system to user devices to be applied by the user devices.

BACKGROUND OF THE INVENTION

Malware (or malicious software) is a computer program that is often designed to disrupt network communications, gain control over computers or networks, or secretly gather personal information about users. Malware typically includes viruses, trojans, adware, and spyware, to list a few examples.

Malware is created for a variety of reasons such as to achieve wide-spread notoriety or for personal gratification. Alternatively, malware is created to secretly access financial information such as banking records, credit card numbers, and/or social security numbers of individuals. While these exploits are frustrating and possibly destructive, they are also fairly simple.

Currently, signature based anti-malware software combats malware by using hashes (i.e., unique signatures of the malware) to identify and quarantine (or remove) the malware. Typically, the anti-malware software utilizes databases containing records of hashes for known malware. If a program's hash matches a hash of known malware, then the program is quarantined or removed.

SUMMARY OF THE INVENTION

One problem with current anti-malware software is that the software is not able to detect new malware threats (often referred to as zero-day or day-zero malware) because hashes for these new threats do not exist in any databases of known malware. Additionally, in some cases, the new malware is able to evade signature based anti-malware software for weeks or even months.

Additionally, as malware evolves, the complexity and purpose of the malware evolves as well. For example, two recent trends in the creation of malware are making existing anti-malware software less effective.

The first trend is that a large percentage of new malware is polymorphic. That is, when the malware replicates, it also mutates to change the contents of the file containing the virus and possibly the behavior of the malware. Thus, each mutation creates a new version of the malware with a new and unique hash. The polymorphic nature of the malware renders traditional signature based solutions ineffective because the newly created hashes do not exist in any databases of known malware.

The second trend is the emergence of advanced persistent threats, which are often implemented by organized crime groups or state-sponsored by foreign entities. The advanced persistent threats are uniquely customized attacks that target individuals or specific companies. The goal of the attack is often the undetected theft of sensitive data, financial information of individuals or companies, or incapacitation of a victim's computer or network. Because the advanced persistent threats are customized to the victims and are designed to be undetected, the signatures of the advanced persistent threats are rarely added to the databases of known malware.

Currently, there are no solutions that adequately address day-zero malware or advanced persistent threat problems in addition to more traditional malware versions. The present invention concerns a method and system that can detect new or polymorphic computer viruses, persistent day-zero exploits, advanced persistent threats, and other malicious software. Additionally, the present invention can be directed to applying monitors and controls to user devices, which are able to protect against these exploits.

In more detail, in a proposed system, a cloud security policy system receives hashes and behavioral information about applications and/or process from different user devices. The cloud security policy system records this information along with a time-stamp to track when an event (e.g., file accessed, created, or loaded) occurred. The cloud security policy system then evaluates the trustworthiness of the hashes based on the information received from the different user devices to provide (or update) a security policy for the applications and/or processes. The security policy is then sent from the cloud security policy system to user devices to be applied by the user devices.

In general, according to one aspect, the invention features a system for detecting malware. The system includes user devices that monitor executing applications and a security policy system that receives requests from the user devices for security policies associated with the applications and sends the security policies to the user devices from which the requests originated.

In general, according to another aspect, the invention features a method for providing security policies. The method includes receiving behavioral information about processes executing on different user devices, determining trustworthiness for each of the processes based on the behavioral information received from each of the different user devices, and providing security policies for the processes to the different user devices based on the determined trustworthiness.

In general, according to still another aspect, the invention features a security policy system. The system includes a web services component of the security policy system that receives behavioral information about processes executing on different user devices. The system further includes an analysis engine of the security policy system that determines trustworthiness for each of the processes based on the behavioral information received from each of the different user devices. Lastly, the system also includes a policy engine of the security policy system that provides security policies for the processes to the different user devices based on the determined trustworthiness.

In general, according to still another aspect, the invention features a method for implementing security policies on user devices. The method includes monitoring processes executing on user devices and searching for security policies associated with the processes. The method further includes upon locating security policies, applying the security policies to the processes, and upon failing to locate security policies on the user devices, sending requests to a security policy system. Lastly, upon receiving security policies from the centralized security system, applying the security policies to the processes.

In general, according to still another aspect, the invention features a method for monitoring applications on user devices. The method includes monitoring applications requesting to open files using system dynamic-link libraries and searching for hashes corresponding to filenames of the files requested by the application in caches of the user devices. The method includes that upon locating hashes in the caches of the user devices, searching for security policies associated with the hashes. Additionally, upon locating the security policies associated with the hashes, enforcing restrictions of the security policies.

In general, according to still another aspect, the invention features a method for monitoring processes executing on user devices. The method includes intercepting application program interface calls to monitor resource requests of executing processes. The method further includes maintaining a log of the resource requests in a database if the processes are being monitored. The method further includes applying security policies to the processes if the processes are controlled by security policies and sending the log of resource requests to a security policy system.

In general, according to another aspect, the invention features a distributed security system for monitoring processes executing on user devices. The system includes an application program interface detour that intercepts application program interface calls and monitors resource requests of executing processes. The system further includes a reputation manager that applies security policies to the processes if the processes are controlled by the security policies. The system further includes a database of a user system that stores logs of resource requests if the processes are being monitored by the reputation manager and a reputation database of a security policy system that stores logs of resource requests from multiple user devices.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1Ais a block diagram illustrating a distributed security system100for the detection and control of malware, which has been constructed according to the principles of the present invention.

In general, the distributed security system100includes user devices102-1to102-nin communication with a cloud security policy system107via a private and/or public data network such as the Internet106. The user devices102-1to102-nmonitor applications and send information and security requests to the cloud security policy system107. The cloud security policy system107determines the trustworthiness for the applications, processes, and files, for example, and provides security policies to the user devices102-1to102-n.

In a typical implementation, each user device102-1to102-nruns agent security software, which monitors applications and/or processes executing on the user devices102-1to102-nalong with access files from storage media and/or via network interfaces. The user devices102-1to102-ninclude desktop and laptop computers (running Windows by Microsoft Corp., Mac OS X by Apple Inc., Linux), tablets or slate computing devices, and mobile computing devices (e.g., smartphones running iOS by Apple Inc. or Android by Google Inc.), to list a few examples.

In a typical implementation, if the agent security software detects an application accessing files on the user device (e.g.,102-1), the agent security software attempts to locate a security policy for the application on that user device. If the agent security software is unable to locate security policy on the user device, then the agent security software sends a request for a security policy to the cloud security policy system107via the Internet106.

The cloud security policy system107receives security policy requests from all the user devices102-1to102-n, calculates trustworthiness of applications and/or files based on information received from the user devices102-1to102-n, and provides customized security policies to the user devices102-1to102-nfrom which the requests originated.

In the illustrated embodiment, the cloud security policy system107includes a web services component108, a policy engine110, and an analysis engine114.

The web services component108receives security policies request from user devices102-1to102-nand forwards the requests to the policy engine110.

The policy engine searches for security policies in the configuration and security policy database112and reputation database116.

The analysis engine114calculates trust (or reputation) scores to determine the trustworthiness of the applications and whether the applications are malicious or benign.

In the illustrated example, the cloud security policy system107also includes a behavioral information database118that stores behavioral information about applications received from user devices102-1to102-nand a whitelist/blacklist database120that stores records of whitelisted and blacklisted applications. In a typical implementation, the databases of the cloud security policy system107(e.g., reference numerals112,116,118, and120) are a SQL (Structured Query Language) databases.

FIG. 1Bis a block diagram illustrating user devices102-1to102-nsending information to a cloud security policy system.

In a typical implementation, the cloud security policy system107utilizes crowdsourcing to collect and analyze information about applications and/or files. That is, information received from all the user devices102-1to102-nis used to determine the security policies for the applications and/or files. In the preferred embodiment, the user devices102-1to102-ncommunicate with the cloud security policy system107via a networking protocol such as TCP/IP (Transmission Control Protocol/Internet Protocol).

In an alternative embodiment, individual messages (e.g. security policy requests and/or response messages) are sent using a stateless communication protocol such as UDP (or User Datagram Protocol), according to one implementation.

In a preferred implementation, the data transmitted between the user devices102-1to102-nand the cloud security policy system107are encrypted and authenticated using common protocols such as SSL (Secure Socket Layer) or TLS (Transport Layer Security). In the case where UDP is used as the network transport, a correspondingly appropriate security protocol such as DTLS (Datagram Transport Layer Security) is used to secure the communications.

FIG. 2is a block diagram of the agent security software architecture that is implemented on the user devices102-1to102-n.

The user devices102-1to102-noperating system is usually divided between a user space and a kernel. Generally, the user space is reserved for user applications and the kernel manages processes, system memory, and hardware components of the user devices102-1to102-n.

The illustrated example is for a Windows-based operating system sold by Microsoft Corp. Different operating systems generally have different kernels and different user spaces. Moreover, even different versions of the same operating system typically have different kernels. Thus, the way in which the kernel interacts with software of the user space and hardware of the user devices102-1to102-nis different. These differences must be factored by the cloud security policy system107because different operating systems, kernels, and user spaces will have different vulnerabilities and malware programs that are dangerous for one may be harmless to another.

In the illustrated example, the user space includes the agent reputation manager204and agent local database206, which manage and store security policies received from the cloud security policy system107. The user space also includes an agent API (application program interface) detour212, that intercepts API calls and resource requests made by applications being monitored (e.g., controlled monitored application214). The agent API detour typically does not intercept application applications that are known (e.g., known application216).

In the illustrated example, the kernel includes an agent file filter208and an agent cache210, which are used to map filenames of applications to corresponding hashes and enforces security policies for applications and processes.

The kernel further includes device drivers, which enable software programs to interact with hardware of the user devices102-1to102-n. For example, a filter manager220provides functionality required by filter drivers to monitor and control resource requests made to the file system. The file system222manages and organizes how data are used by the operating system. Some examples of file systems for Windows operating systems include File Allocating Table (FAT32) and New Technology File System (NTFS), to list some examples. Filter drivers are often optional drivers that are able to modify the behavior of a device. On user devices running an operating system such as Windows 7, filter drivers can be implemented for both file and network access. In the case where access control or content filtering needs to be performed on a file, the filter driver sits between an application and the underlying file system and has the ability to scan or restrict access to files based upon enforced security policy. For example, the file filter driver prevents an application (or executable) from being read, loaded into memory or executed if the file hash been identified as being malware in one example. This is accomplished by returning an “Access Denied” status code to the calling application when an attempt was made to access the resource, in one example.

In one specific example, Microsoft Corp. has provided sample code (via the MSDN and the WDK documentation), which implements a variety of File System Minifilter Drivers. One such example, the SCANNER minifilter explains how a filter driver can detect a file access or file creation, scan the contents of the data looking for a “sample virus pattern” and report relevant information to a user level service daemon. This example shows how anti-virus/malware software can detect file access and scan the contents for virus signatures.

A TCP/IP driver224enables the user devices102-1to102-nto have network access. The kernel further includes a filtering platform226, which is a set of API and system services that provide features that can be used by packet processing or other connection monitoring services (e.g., firewalls). The kernel also includes an agent network filter228, which is able to monitor and track all network connections made on a per process basis. If the application file's hash was flagged by a security policy, network connections may be denied or terminated on a per process basis by returning a failure code to a caller indicating the request to access the resource (in this case the network) is denied (or blocked).

In one embodiment, content filtering on the network is used to block or filter spam, inappropriate web-sites or content, and malware being downloaded. Generally, anti-virus software is a form of content filtering because the software scans binary attachments in mail or files downloaded via the web and tries to find known virus signatures. Additionally, content filters may be implemented via software on individual computers or at a central point on the network, such as a firewall, internet router, or proxy server. Apache (by the Apache Software Foundation) is a commonly used, open sourced web server which may act as a proxy server and supports filtering. As data passes through a filter, a cryptographic hash is calculated for the data stream or attached file.

FIG. 3Ais flow diagram illustrating the steps performed by the agent file filters208of the user devices102-1to102-nto monitor applications executing on the user devices102-1to102-n.

In general, the agent security software is responsible for detecting new files being created or accessed on the user devices102-1to102-n. In a preferred embodiment, the agent security software implements utilizes the filter manager220, which is capable of detecting a new file being created or files being accessed. The filter manager notifies the agent file filter, which reads the file and calculates a hash to uniquely identify the file. There are several well known cryptographic hashes such as MD5, SHA-1, and SHA-256, which are known in the art. A hash (or cryptographic) hash is a one-way deterministic function that takes an arbitrary stream of data (or message) and returns a fixed-sized string (a message digest or hash). Different streams of data always result in different and unique hashes, but the same stream or message always yields the same hash. This is important because filenames cannot always be relied upon to accurately identify a file.

Hash functions are often used for information security, providing integrity checks of data/information and providing digital signatures of the data, to list a few examples. Hashes have several useful characteristics such as it is not feasible to modify a message without changing the hash, it impossible or at least very improbable to find two different messages with the same hash, and it is generally not possible derive the original message from the hash.

Additionally, reference implementations or binaries are also available to uniquely identify applications.

In the first step302, an application executing on a user device (e.g.102-1) makes a request to open a file via the system DLL (API). In the next step304, the filter manager220notifies the agent file filter208of the request to open the file. Next, the agent file filter208looks up the filename of the file in the agent cache210in step308.

If the filename is not mapped to a hash stored in the agent cache210, then the agent file filter208scans the contents of the file and calculates a hash for the file in step312. In the next step314, the agent file filter208updates the agent cache210by adding the filename/hash to agent cache210.

If the filename of the file is mapped to a hash in the agent cache210or the agent file filter208updated the cache by adding the filename to the hash, then the agent file filter208looks up a security policy for the hash in the agent cache210in step316.

In the next step318, the agent file filter208determines if the security policy is in the agent cache210. If the security policy is in the agent cache210, then the agent file filter208enforces the specified restrictions of the security policy in step320.

If the security policy is not in the agent cache210, then the agent file filter208sends a message via a user/kernel communication channel to the agent reputation manager204requesting the security policy for the hash in step322. In the next step323, the agent file filter208stalls while waiting for a response from the agent reputation manager204.

In the next step324, the agent file filter208determines if the response from the agent reputation manager is timely. If the response from the agent reputation manager204is not timely, then the agent file filter208enforces a default security policy (e.g., “Fail Open”) in step326. If the response from the agent reputation manager204is timely, then the agent cache210is updated with the security policy from the agent reputation manager204in step328. In the next step320, the agent file filter208enforces the security policy enforcement actions of the security policy.

FIG. 3Bis flow diagram illustrating how security policies are enforced on the user devices by the agent file filter208(step320ofFIG. 3A).

In the first step330, the file filter208determines if the security policy denies file access. If the security policy denies file access, then the agent file filter208denies the application file access in step332. If the security policy does not deny file access, then the agent file filter208allows file access in step334.

FIG. 3Cis flow diagram illustrating the steps performed by an agent reputation manager204to locate and enforce a security policy for an application executing on one of the user devices (step322ofFIG. 3A).

In the first step340, the agent reputation manager204receives security policy requests from the agent file filter208. In the next step341, the agent reputation manager204searches the agent local database206for the requested security policy of the hash. In the next step342, the reputation manager determines if the security policy is in the agent local database206. If the security policy is in the agent local database206, then the reputation manager sends the security policy for the hash to the agent file filter208in step344.

If the security policy is not found in the agent local database206, then the reputation manager204sends a message to the cloud security policy system107requesting a trust score (or reputation score) for the hash of application in step346. In the next step348, the reputation manager204determines if a response from the cloud security policy system107is received within a predetermined length of time. If there is no response from the cloud security policy system107within a predetermined length of time, then the agent file filter208enforces a default security policy for the application in step356. The default security policy provides enforcement actions and application restrictions for monitored applications.

In the next step358, the agent file filter208stores a record of the failed response from the cloud security policy system107. Next, in step360, the agent reputation manager204waits a predetermined length of time and resends the request to the cloud security policy system107.

If the response from the cloud security policy system107is received within the predetermined length of time, then the agent reputation manager204maps the received trust score to the security policy in step350. In the next step352, the agent local database206is updated with the security policy. In the final step344, the response is sent to the agent file filter210, which updates the agent cache210and enforces the security policy.

FIG. 4is a flow diagram illustrating the steps performed by the cloud security policy system107to handle requests from the user devices102-1to102-n.

In the first step1002, the user devices102-1to102-nsend reputation requests for hashes to the cloud security policy system107. In the next step1004, the reputation requests are received by the web services component108. The web services component108forwards the reputation requests to the policy engine110in step1006. In the next step1008, the policy engine110searches for the hashes in the reputation database116.

In the next step1010, the policy engine110determines if entries for the hashes exist in the reputation database116. If the entries do not exist in the reputation database116, then the policy engine110instructs the analysis engine114to create new entries in step1012. Next, in step1014, the analysis engine114searches the whitelist and blacklist database120and behavioral history database118for additional information about hashes for additional information that can be used to calculate trust scores for the hashes.

In the next step1016, the analysis engine114calculates the trust scores for the hashes. In the next step1018, the analysis engine updates the entries in the reputation database116with the trust scores. In the next step1020, the policy engine110retrieves the entries from the reputation database116.

If the entries exist in the reputation database116(or after the policy engine110retrieves the entries from reputation database in step1020), the policy engine110searches for the user devices102-1to102-nin the configuration and security policy database112in step1026. In the next step1028, the policy engine110identifies the user devices102-1to102-nand a customer associated with each user device. In the next step1030, the policy engine110retrieves the security policies for each user device. Next, in step1032, customer specific mapping of the trust scores are checked to determine the security policy enforcement actions of the security policies. In a typical implementation, the enforcement actions of the security policies are user device (or customer) specific. Thus, identical trust scores result in different enforcement actions for different user devices102-1to102-n, in one implementation.

In the next step1034, the trust scores and security policies are sent to the web services component108. In the next step1036, the trust scores and security policies are sent back to the user devices102-1to102-n. Lastly, the user devices102-1to102-nstore the user specific security policies and trust scores in the agent local databases206and the agent caches210in step1038and then enforce those security policies.

FIG. 5Ais a table illustrating an example of mapping between unknown applications/hash behaviors and default security policy enforcement actions.

In the illustrated example, security policy enforcement actions402are mapped to behaviors of unknown applications/hashes404. In a typical implementation, the behaviors and corresponding enforcement actions are customizable for different user devices. For example, if an unknown application turns on a microphone or monitors keystrokes of user devices102-1to102-n, then the corresponding enforcement action implemented by the agent security software is to terminate the application. This is because the unknown application is performing actions that are typically performed by malicious software or malware.

In another example, if an unknown application attempts to read user documents (which could be malicious or benign), then the corresponding enforcement action is to prevent network access for the unknown application. Lastly, other actions such as reading user contact information are not restricted in any way.

FIG. 5Bis a table illustrating an example of mapping between user specific trust scores and security policy enforcement actions.

In the illustrated embodiment, default security policy enforcement actions406are based on trusts scores408of the applications/hashes. Rather than having security policy enforcement actions for specific behaviors, the security policy enforcement actions correspond to calculated trust scores of the applications/hashes.

For example, trust scores between 0.0 and 3.0 result in termination of the application. Trust scores between 3.1 and 4.9 result in prevention of network access for the unknown application. And trust scores greater than 9.0 result in no restrictions for the unknown application.

In a typical embodiment, security policy enforcement actions are also based on crowdsourcing, which helps reduce false positives by collecting and analyzing behavioral information from a large number of user devices (or companies). The cloud security policy system is able to identify unknown and/uncommon applications (based upon crowdsourcing as well as a centralized list of known applications). This allows it to reduce False Positives when analyzing behaviors and in applying security. Using this approach, the system can identify the good applications and focus on the unknown or bad ones.

For example, if only one company is reporting (or requesting trust scores) about an unknown applications, the security policy enforcement action is to terminate the application. In another example, if the total number of companies reporting about an unknown application is 3 or less, then the application associated is not permitted to access user documents or sensitive data (e.g. database files or financial records). In another example, if unknown applications are not widely used (e.g. on fewer than 1000 user devices) across all companies, the applications are prevented from accessing the network according to one policy example. In another example, if the age of the unknown application is less than 1 week, then the application is not able to access any networks or make any network connection. Additionally, this crowdsourcing information is also used to calculate the trust scores for the applications.

FIG. 6Ais a flow diagram illustrating an example of how trust score are calculated for unknown applications/hashes.

Trust scores are numeric representations used to determine the trustworthiness of unknown applications. In a current implementation, the higher the trust score, the more trustworthy an application is considered. Conversely, lower trust scores indicate a greater chance that the application is malware. In the illustrated example, the scale is from 0-10 (with one decimal place). Alternate embodiments, however, could implement different scales with greater or finer increments and/or utilize a larger or smaller scale.

In the illustrated example, combinations of observed behaviors and/or the absence of expected behaviors are used to calculate the trust score for unknown applications. For example, in the first step502, the unknown application (or its hash) is assigned an initial score of 0.0. In the next step504, the analysis engine114determines if the unknown application is known/good. If the application is a known/good application, then the analysis engine114determines if the application is widely used in step506. Additionally, if the application is known/good, then the application is no longer “unknown”, but application could still be malware. Thus, if the application is not widely used, then the unknown analysis engine114assigns the application a score of 5.0 in step510.

Alternatively, if the application is widely used, then the analysis engine114assigns the application a score of 9.0 in step508. The discrepancy in trust scores for known applications that are widely used versus known applications that are not widely used is because it is possible for malware to be added onto a Whitelist (e.g. “gaming” the system and compromising the whitelist database). The different scores for how widely used the application is provides a greater chance of detecting whether the application is malware.

If the unknown application is not known/good, then the analysis engine114determines if the unknown application read user data in step512. If the unknown application did not read user data, then the analysis engine114assigns a score of 6.0 in step514. If the unknown application reads user data, then the analysis engine114determines if an HTTP (Hypertext Transfer Protocol) connection was made in step516. If the unknown application did not make an HTTP connection, then the analysis engine114assigns a score of 6.0 in step518. If the unknown application makes an HTTP connection, then the analysis engine114determines if a visible window is displayed in step520.

If the unknown application did display a visible window, then the analysis engine114assigns a score of 4.5 in step524. If the unknown application did not display a visible window, then the analysis engine114assigns a score of 2.0 in step522. This is an example of how the absence of an expected behavior causes the trust score to be affected (e.g., lowered).

In alternative embodiment, the analysis engine114could also check to see if the application performed other actions such as turning on microphones, turning on webcams, accessed databases, or recorded keystrokes, to list a few examples.

Additionally, points may be added to the trust scores based on other factors such as the age of the application (how long the application has existed), the number of devices and/or companies reporting on the application, static information about the file or code (e.g. filename, publisher, and whether the application is signed), to list a few examples. Additionally, contextual information about the application file can also affect the trust score. For example, what application created or downloaded the file, where the file was downloaded from (e.g., from USB, from a remote network peer, or overseas website), and the behavior exhibited by the code of the application as it executes on the user devices102-1to102-n(e.g. network connections, system API calls, files accessed, and user inputs monitored), to list some examples.

Other methods may be used to calculate the trust score. By way of a simple example, if there is only one company reporting on the application, then zero points are added to the trust score. If the number of companies reporting an application is 10 or less, then add 2.5 points to the trust score. If the number of companies reporting on the application is greater than 10, then add 5.0 points to the trust score.

In another example, if the application's age is less than 1 day, then add zero points to the trust score. If the application's age is less than 1 week, then add 0.5 points to the trust score. If the application's age is less than 1 month, then add 1.0 points to the trust score. If the application's age is greater than a month, then add 3.0 points to the trust score.

In yet another example, if there is only one user device reporting the application, then add zero points to the trust score. If the number of user devices102-1to102-nreporting the application is 1000 or less, then add 1.0 points to the trust score. If the number of user devices102-1to102-nreporting the application is greater than 1000, then add 2.0 points to the trust score.

In an alternative embodiment, other methods to calculate the trust score are implemented. In one example, the trust score is statically assigned by a data flow terminator or incrementally modified as each node is evaluated in a decision tree. In this embodiment, the trust scores may be a fixed value, variable value, dependent upon the current node in the data flow, dependent upon the behavior being evaluated at that point in the decision process, or dependent upon the probability that the intent is malicious or not (e.g., via a Bayesian Network).

In the preferred embodiment, observed behaviors of the applications are run through multiple and possibly different behavioral models designed to search for different behaviors. The different behavioral models search for behaviors such as the “intent” of the applications, data theft by the applications, or indicators that the application operating as part of a botnet, to list a few examples. If multiple trust scores are calculated for the application, then the analysis engine114is able to choose which trust score to use when selecting a security policy for an application. Typically, the analysis engine114selects the lowest calculated trust score.

For example, when the behaviors of the applications are modeled against botnet behaviors, the behaviors of the applications may not exhibit the behaviors of a botnet and thus receive a higher trust score. When the behaviors of the applications are modeled against behaviors of data theft, the behaviors may match the behaviors of data theft and thus receive a lower score. Thus, the analysis engine114selects the lower score to indicate a higher possibility of malware directed to data theft.

In the preferred embodiment, trust scores score are generated for specific behaviors on a single user device or for behaviors on an aggregate set of user devices (i.e. where an application running on multiple user devices exhibits a specific type of behavior). If the trust score is generated for the aggregate set of user devices, then the behaviors do not need to occur on every user device. The behaviors only need to occur on enough user devices for the cloud security policy system107to determine that the behavior is representative. This results in both an “incident” score for the specific devices and well as a “collective” trust score based on aggregated data. To provide an example, the analysis engine114typically selects the lower score for policy enforcement. In another embodiment, both the “incident” and “collective” scores are available for use in enforcing security policies. Typically, multiple sets of different user devices102-1to102-nare used in calculating more than one aggregate or “collective” trust score. The user devices102-1to102-n, which are evaluated in a specific aggregation of data, typically include a single company, a vertical or companies in a similar business, an arbitrary collection of users or companies, a geographic collection of user devices, or a global collection of user devices, to list a few examples.

In the preferred embodiment, the trust scores for the applications running on the user devices102-1to102-nhelp calculate the trust score for the user device itself. This score represents the “trustworthiness” of the device itself (in totality). Additionally, this score may be used in controlling access to cloud or network based services, admission to specific networks, compliance enforcement, or risk rating, to list a few examples.

FIG. 6Bis a table illustrating how actions of unknown applications are compared to actions performed by malware and trusted applications to determine the trustworthiness of the unknown applications.

In some embodiments, the cloud security policy system107uses statistical analysis to identify behavior exhibited by malware. In the illustrated example, a set of “control” data is identified, which may be common to both a trusted applications and malware. In this case, both applications may, for example, “read user data” (e.g. a word document) and make a network connection using HTTP.

Another set of data (the “variance” data set) identifies behavior of the applications, which statistically vary (or are measurably different) between trusted applications and malware. The behavior is typically a combination of observed behaviors as well as the absence of expected behaviors. In the illustrated example, the behavior is whether or not the application displayed a “visible window.” For trusted applications, a “visible window” was display 98% of the time. In the case of malware, a “visible window” was displayed only 15% of the time.

Examining “Unknown App 1”, the behaviors of the unknown application match the behavior of the control data. Looking at the variance behavior, a “visible window” was not displayed. And given the statistical divergence between the trusted applications and malware, the “unknown app 1” is more likely to be “Malware” than a trusted application because its behavioral profile is similar to malware.

In addition to the “control” and “variance” data sets, the behavior model also collects additional “unmodeled” data about the applications. The additional “unmodeled” data includes network connections, types of file access and/or creation, changes to the system configurations (e.g. Windows registries), and system or application API calls, to list a few examples. While these behaviors may or may not be malicious, monitoring which applications exhibit these behaviors help the model to evolve over time with the malware. In one embodiment, new malware behavior is learned and malware applications are identified based upon the “variance”. The “unmodeled” behavior which has also been collected may then be compared between Trusted applications and known malware to detect other behavioral outliers. For example. calls to a specific system API (e.g., SysAPI call Y) are statistically similar with malware and the trusted application. Thus, there is nothing to distinguish a trusted application from malware (using that control set). Therefore, whether the application makes SysAPI call Y will not factor into trust scores, in one example. Conversely, modifications to a specific Registry Key Value (Key X), are more likely in Malware than in a trusted application. This learned behavior may transition and be included as a known “variance” to identify Malware (using this control set) in future analysis. Thus, the model is able to evolve over time with evolving malware.

Examining “Unknown App 2”, the behaviors of the unknown application match the behavior of the control set. Looking at the variance, a “visible window” was displayed. Thus, the application initially appears to be a trusted application. However, if the “learned” behavior (modification of the “Registry Key X”) is factored in as a “variance”, then the analysis may indicate that the “Unknown App 2” is more likely to be malware.

In some cases the mere presence of variance data (irrespective of the control set) may be indicative of malware. For example, the modification of Registry Key X combined with an HTTP connection typically indicates malware. Thus, regardless of other behaviors, if an unknown application performs these behaviors it is determined to be malware.

Alternatively, the combination of control and variance behavior is could be viewed as a new behavioral model, where both behaviors are considered the “control” set of data indicating malware.

In an alternative embodiment, a set (or a subset) of behaviors exhibited by a specific instance of malware are used to identify polymorphic variations of the specific instance malware. That is, observed behavior of malware is able to be used as a fingerprint to identify the malware. In the case of polymorphic viruses, files that appear to be different (based upon a hash of the file) will exhibit the same set of observed behaviors as the malware. Thus, the set (or subset) of behaviors is able to identify the polymorphic virus. Additionally, behavioral fingerprints are also able to identify unique applications (e.g. applications that only occur on a single device) which exhibit the same behavior and are likely malware.

FIG. 7is a flow diagram illustrating the steps performed by the agent file filter208during background scans of the user devices102-1to102-nto determine if files have been created, deleted, or modified.

If files have been created or modified, then it is likely that their corresponding filenames and hashes have also changed. Additionally, the agent cache210(possibly the agent local database206) needs to be updated to reflect these changes.

In the first step602, the agent file filter208determines if files have been deleted. If the files have been deleted, then corresponding entries are removed from the agent cache210in step604. If files have not been deleted, then the agent file filter208determines if any files have been created or modified in step606.

If any files have been created or modified, then the agent file filter208scans (or rescans) the created or updated files in step608. In the next step610, the agent file filter208updates the entries in the agent cache210and maps the filename of the updated files to the hash in the agent cache210. In the next step612, the agent reputation manager204forwards static information about the files to the cloud security policy system107. Static information often includes the size of the file, whether the file/executable is “packed”, or whether the file/executable is “signed” (and by what certificate authority), to list a few examples.

In alternative embodiments, other methods for detecting new (or updated) files on user devices102-1to102-nare implemented. For example, application plugins, which are often present in web browsers, are able to detect when a file is being downloaded. Alternatively, another method includes using a service daemon, which performs a background scan of the disk, to search for files which have modified time-stamps or file sizes.

FIG. 8is flow diagram illustrating the steps performed by the agent reputation manager204to monitor processes executing on the user devices102-1to102-n.

In the first step704, the agent file filter208is notified by the operating system that a new process was created. In the next step706, the agent reputation manager204is informed by the agent file filter210of the new process. In the next step708, the agent reputation manager204identifies the filename of the process by making API calls. Next, in step710, the agent reputation manager204sends a request to the agent file filter210for the hash of the filename of the created process. In the next step711, the agent file filter208searches for the hash in the agent cache210. If the agent file filter208is not able to locate the hash in the agent cache210, then the agent reputation manager204scans the file to calculate a hash in step713. If the agent file filter208is able to locate the hash in the agent cache, then the agent reputation manager204retrieves the hash information from the agent cache210in step712.

In the next step714, the agent reputation manager204searches the agent local database206for a security policy corresponding to the hash, which is retrieved from the agent cache210. In the next step716, the agent reputation manager204determines if the security policy was fond in the agent local database206. If the agent reputation manager204locates the security policy in the agent local database206, then the agent reputation manager204informs the agent file filter208and the agent network filter228to enforce the security policy in step728.

If the agent reputation manager204is not able to locate the security policy in the agent local database206, then the agent reputation manager204enforces a default security policy in step718. In the next step720, the reputation manager204sends a request for a security policy (corresponding to the hash) to the cloud security policy system107. In the next step720, the reputation manager204updates the agent local database206if the cloud security policy system107responds with a security policy.

In the next step724, the agent cache210is updated with the security policy. Next, in step726, the agent file filter208and agent network filter228are updated with the security policy.

FIG. 9is flow diagram illustrating how processes are monitored and/or controlled by the reputation manager204.

In the first step802, the agent API detour212is loaded as a DLL into the process. In the next step804, the agent API detour212sends a message to the agent file filter208for a security policy associated with the process. The agent file filter208then determines if the process is monitored or controlled in step806. If the process is not monitored or controlled, then the agent API detour212will not intercept or modify process behaviors in step808. In the next step810, the agent API detour DLL may be unloaded from the process.

If the process is monitored or controlled, then the agent API detour212intercepts API calls from the process in step812using software such as the Microsoft Detours product. In the next step814, the agent API detour212intercepts resource requests from the process.

Next, in step815, the agent file filter208determines if the process is being monitored. If the process is being monitored, then the agent API detour212logs requested API calls to agent reputation manager204. In the preferred embodiment, this is accomplished by sending a message via the agent file filter208. In other embodiments, the monitored information is passed directly to the reputation manager204. Alternatively, the monitored information is stored in a log file, syslog, or an NtEvent Log, to list a few examples. Then the monitored information is read in by the reputation manager. In the next step, the reputation manager204stores the monitored information to agent local database206. Then, the agent reputation manager204sends the monitored information to cloud security policy system107in step828.

In the next step818, or if the process is not being monitored (from step815), then the agent file filter208determines if the process is being controlled based on a security policy. If the process is not being controlled based on the security policy, then agent API detour intercepts resource requests in step814. If the process is being controlled based on a security policy, then the restrictions within the security policy are applied to the resource requests in step820. The restrictions include the agent network filter228preventing network connections, in one implementation. Alternatively, the process being controlled is terminated (in the case where the security policy indicates the application is not allowed to run). Typically, the application is terminated by either the reputation manager204or the agent file filter208terminating the process (operating in a privileged context). Alternatively, the application is terminated by indicating the security policy to the agent API detour212, which causes the process to exit.

FIG. 10Ais a flowchart illustrating the steps performed by the agent security software of the user devices102-1to102-nto check for security policies from the cloud security policy system107at predefined intervals.

In the first step904, the agent reputation manager204determines if “CheckForPolicy” equals TRUE. If “CheckForPolicy” does not equal TRUE, then the agent reputation manager204waits a predetermined length of time in step906. In the next step908, the agent reputation manager204determines if “TimeSinceLastCheck” is greater than the specified timeout period. If “TimeSinceLastCheck” is greater than the timeout period, then the reputation manager204sets “CheckForPolicy” to TRUE and returns to step904. If “TimeSinceLastCheck” is not greater than the timeout period, then the reputation manager returns to determine if “CheckForPolicy” equals TRUE in step904.

Returning to step904, if “CheckForPolicy” equal TRUE, then the agent reputation manager204sends a request for a security policy to the cloud security policy system107in step912. In the next step914, the web services component108receives the request for the security policy from the user device. In the next step916, the web services component108forwards the request to policy engine110. In the next step918, the policy engine110looks up the user device in configuration & security policy database112.

Next, the policy engine110retrieves the default security policy for the user device in step920. In the next step922, the web services component108sends the security policy to the user device. In the next step924, the agent reputation manager204stores the security policy in agent local database206. In the next step926, the agent reputation manager sets “CheckForPolicy” to FALSE and updates “TimeSinceLastCheck”. The agent security software records the time for “TimeSinceLastCheck”, so it is able to calculate when then next timeout period should expire, causing the agent security software to recheck for a new security policy.

FIG. 10Bis a flowchart illustrating the steps performed by the agent security software of the user devices102-1to102-nto check for security policies from the cloud security policy system107based on messages or responses received from cloud security policy system107.

In the first step950, the agent reputation manager204sends a message to cloud security policy system107. In the next step952, the agent reputation manager204receives a response from cloud security policy system107. In the next step954, the agent reputation manager204determines if the response indicates that security policy has changed.

If the security policy received from the cloud security policy system107has changed, then the reputation manager204sets “CheckForPolicy” equal to TRUE in step956. In the next step958, or if the response did not indicate that security policy has changed, then the reputation manager processes the response received from cloud security policy system107.

The remaining steps960-982are identical to corresponding steps904-926ofFIG. 10A. In the preferred embodiment, step960is processed by a separated worker thread running in the reputation manager204.

FIG. 11is flow diagram illustrating how log information is sent from user devices102-1to102-nto cloud security policy system107.

In the first step1102, the user devices102-1to102-nsend report messages with report logs about application behaviors to the cloud security policy system107. In the next step1104, the web services component108receives the report messages. In a typical implementation, when the web services component108receives report messages of new hashes from the user devices102-1to102-n, the web services component108includes a time-stamp to record when the message arrived.

In the next step1106, the web services component108forwards the report messages to analysis engine114. The analysis engine114stores the report message information in the behavioral history database118in step1108. Additionally, after each report message is received, the behavioral history database118is updated to reflect how many different user devices102-1to102-n(or companies) have reported this hash.

In some embodiments, the database behavioral history database118also includes a summary record for each hash, which includes a set of information that can be used for enforcing security via access control lists (ACLs) by the agent security software residing on the user devices102-1to102-n. In one implementation, the summary records include a time-stamp of when this hash was first reported to the server, a global “age” for the hash, a count of the number of user devices102-1to102-nthat have accessed the hash, a count of the number of organizational units or companies (in a multi-tenant environment) that have accessed the hash, and the trust score of the hash, to list a few examples.

In the next step1110, the analysis engine114searches the whitelist/blacklist database120and the behavioral history database118for other information about hash. Next, in step1112, the analysis engine114calculates a trust score for the hash based on the received information and any information found in the whitelist/blacklist database120and the behavioral history database118. In the next step1114, the analysis engine114updates the reputation database116.