Patent Description:
In many computers and network systems, multiple layers of security apparatus and software are deployed in order to detect and repel the ever-growing range of security threats. At the most basic level, computers use anti-virus software to prevent malicious software from running on the computer. At the network level, intrusion detection and prevention systems analyze and control network traffic to detect and prevent malware from spreading through the network.

<CIT> discloses techniques for the classification of software modules, including potentially malicious software modules such as malware. It is concerned with classifying software as benign or malicious based on comparing features extracted from the software module with features of clusters of previously classified software modules.

<CIT> relates to the detection of a malicious application installed on a mobile device.

The description above is presented as a general overview of related art in this field and should not be construed as an admission that any of the information it contains constitutes prior art against the present patent application.

The invention is defined according to a method claim <NUM> and corresponding independent apparatus claim <NUM> and computer software claim <NUM>. Further details are defined in the dependent claims.

The disclosure is herein described, by way of example only, with reference to the accompanying drawings, wherein:.

One level of security that enterprises can employ to protect their data is to limit software applications deployed on their networks to signed software applications. However, there may be instances where malicious code is inserted into a given software application prior to the application being signed. In these instances, a vendor may distribute, to enterprise customers, signed versions of software applications that include malicious code that can be used to launch a cyberattack on a given customer. These cyberattacks are sometimes known as supply chain attacks.

Embodiments of the present invention provide methods and systems for identifying signed software applications comprising malicious code. As described hereinbelow, multiple host computers executing respective instances of a specific software application are identified, each given instance on each given host computer comprising a set of program instructions loaded, by the host computer, from a respective storage device. Information on actions performed by the executing instances is collected from the host computers, and features are computed based on the information collected from the multiple host computers. The collected information for a given instance is compared to the features so as to classify the given instance as benign or suspicious, and an alert is generated for the given instance only upon the given instance being classified as suspicious.

Since systems implementing embodiments of the present invention can monitor millions of actions performed by dozens of different software applications executing on thousands of host computers deployed at hundreds of enterprises. This wealth of information enables these systems to identify anomalous (and therefore suspicious) actions performed by a given instance of a given application, even if the given application is signed.

<FIG> is a block diagram that schematically shows an example of a security server <NUM> that is configured to train and deploy an anomalous action detection model <NUM>, in accordance with an embodiment of the present invention. In embodiments described hereinbelow, security server <NUM> is configured to train model <NUM> by analyzing an action log <NUM> that stores information collected from actions <NUM> performed by software applications <NUM> executing on host computers <NUM> deployed in a plurality of sources <NUM>.

In some embodiments a set of host computers <NUM> may comprise all host computers <NUM> in all sources <NUM>. For each given software application <NUM>, a first subset may comprise host computers (i.e., in all sources <NUM>) executing respective instances of the given software application, and each given source <NUM> may have a respective second subset of the host computers comprising the host computers in the given source executing respective instances of the given software application.

In some embodiments, each given host computer <NUM> can execute a respective instance of an endpoint agent <NUM> that detects actions <NUM> performed by software application <NUM> executing on the given host computer, extracts information from each given detected action <NUM>, and conveys the extracted information for detected actions to security server <NUM>.

Each source <NUM> can be referenced by a respective source identifier (ID) <NUM>, and may comprise an organization or an enterprise (that has a local data network such as local area network (LAN) <NUM> coupling the host computers (to each other and) to a gateway <NUM> that couples LAN <NUM> to a public network such as Internet <NUM>.

In the configuration shown in <FIG>, host computers <NUM> can communicate, via Internet <NUM> with one or more remote servers <NUM> (e.g., web servers, CDN server, etc.) having respective attributes such as an Internet Protocol (IP) address <NUM>, a domain (name) <NUM>, and an autonomous system number (ASN) <NUM>.

<FIG> is a block diagram showing an example of hardware, software and data components in a given host computer <NUM>, in accordance with an embodiment of the present invention. The given host computer may comprise a host processor <NUM>, a host memory <NUM>, a storage device <NUM>, and a host network interface controller (NIC) <NUM> that couples the given host computer to its respective LAN <NUM>.

In some embodiments, each host computer <NUM> may comprise (or be assigned) a respective host ID <NUM>. Examples of host IDs include, but are not limited to, a media access control (MAC) addresses and local IP addresses.

Storage device <NUM> typically stores a set of files <NUM>. In some embodiments, a given file <NUM> may comprise a respective file signature <NUM> (e.g., a computed hash) and a respective file signature ID <NUM> indicating an identity of an entity that generated the respective file signature. Each given file <NUM> comprises a respective file name <NUM>, a respective file size <NUM>, and a respective file type <NUM>. Examples of file types <NUM> include, but are not limited to, executable, shared library (e.g., a DLL), document, image and ZIP™ (i.e., compressed).

In embodiments herein a given file <NUM> having a given type <NUM> may be referred to by its respective type <NUM>. For example, a given file <NUM> whose respective type <NUM> is "executable" may be referred to herein as a given executable file <NUM>, and a given file <NUM> whose respective type <NUM> is "shared library" may be referred to herein as a given DLL file <NUM>.

In some embodiments a given endpoint agent <NUM> executing on a given host computer <NUM> can classify the file type for a given file <NUM> by analyzing data that the respective host processor reads from or writes to the given file. For example, if the data comprises compressed image data, then the given endpoint agent can classify the file type as an image file type (e.g., JPG).

In the configuration shown in <FIG>, memory <NUM> comprises an operating system <NUM>, management information <NUM>, a registry <NUM>, endpoint agent <NUM>, and a plurality of processes <NUM> having respective process names <NUM>. In some embodiments operating system <NUM> may comprise a set of system calls (syscalls) <NUM> having respective syscall names <NUM>. One example of operating system <NUM> is WINDOWS™, produced by MICROSOFT CORPORATION, Redmond, Washington USA. On example of endpoint agent <NUM> is CORTEX XDR™ produced by PALO ALTO NETWORKS, INC. , of <NUM> Tannery Way, Santa Clara, CA <NUM> USA.

To manage operation of the given host computer, processor <NUM> can execute operating system <NUM>. Registry <NUM> may comprise a database of registry keys <NUM> having respective key names <NUM> that store low-level settings for the operating system. In operation, operating system <NUM> can access (i.e., read from or update) keys <NUM> so as to manage the given host computer.

A given process <NUM> comprises a set of program instructions <NUM> that can be executed by processor <NUM>. To start executing a given software application <NUM>, processor <NUM> can load a given executable file <NUM> (i.e., a given file <NUM> whose respective file type <NUM> is "executable"), and start executing, as a given process, program instructions <NUM>.

In some embodiments, a given process <NUM> may comprise a respective process signature <NUM> and a respective process signature ID <NUM> indicating an identity of an entity that generated the respective process signature.

<FIG> is a block diagram showing an example of hardware and data components in security server <NUM>, in accordance with an embodiment of the present invention. In the configuration shown in <FIG>, security server <NUM> comprises a server processor <NUM>, a server memory <NUM> that stores action log <NUM> and model <NUM>, and a server NIC <NUM> that couples the security server to Internet <NUM>.

In some embodiments, action log <NUM> comprises a set of log entries <NUM>, and model <NUM> comprises a set of features <NUM>. Log entries <NUM> are described in the description referencing <FIG> hereinbelow, and features <NUM> are described in the description referencing <FIG> hereinbelow.

Processors <NUM> and <NUM> comprises a general-purpose central processing units (CPU) or special-purpose embedded processors, which are programmed in software or firmware to carry out the functions described herein. This software may be downloaded to host computer(s) <NUM> or security server <NUM> in electronic form, over a network, for example. Additionally or alternatively, the software may be stored on tangible, non-transitory computer-readable media, such as optical, magnetic, or electronic memory media. Further additionally or alternatively, at least some of the functions of processors <NUM> and <NUM> may be carried out by hard-wired or programmable digital logic circuits.

Examples of memories <NUM>, <NUM> and storage device <NUM> include dynamic random-access memories, non-volatile random-access memories, hard disk drives and solid-state disk drives.

In some embodiments, tasks described herein performed by processors <NUM> and <NUM> may be split among multiple physical and/or virtual computing devices. In other embodiments, these tasks may be performed in a managed cloud service.

<FIG> is a block diagram showing an example of data that processor <NUM> can store in a given log entry <NUM>, in accordance with an embodiment of the present invention. In embodiments herein, log entries <NUM> have a one-to-one correspondence with actions <NUM>, and upon receiving, from a given endpoint agent <NUM> executing on a given host computer <NUM>, information for a new given action <NUM>, processor <NUM> can add a new log entry <NUM>, and populate the new log entry with information such as:.

<FIG> is a block diagram showing an example of data components that the security server can store in model <NUM>, in accordance with an embodiment of the present invention. In addition to the set of features <NUM>, model <NUM> also comprises a set of weights <NUM> having a one-to-one correspondence with the features. In <FIG>, features <NUM> and weights <NUM> can be differentiated by appending a letter to the identifying numeral, so that the features comprise features 108A-108O, and the weights comprises 130A-130O.

In embodiments herein, processor <NUM> computes, for each given software application <NUM>, features <NUM> that reflect respective metrics of the actions performed by the given software application. Additionally, the features may comprise:.

Examples of global features <NUM> that processor <NUM> can compute, based on log entries <NUM>, include:.

Examples of local features <NUM> that processor <NUM> can compute, based on log entries <NUM>, include:.

In addition to global and local features <NUM> described hereinabove, features <NUM> may comprise feature 108O that is a hybrid (i.e., global/local). For each combination comprising a given distinct source ID <NUM> and a distinct normalized application ID <NUM> (i.e., referencing a given software application <NUM> executing on one or more of the host computers at a given source <NUM>), processor <NUM> can compute a respective count of distinct normalized actions <NUM> (i.e., performed by the given software application on a given host computer <NUM> at the given source). For each given computed count, processor <NUM> can compute a respective feature 108O by averaging all the computed counts other the given computed count, and then comparing the given computed count to the computed average. This can be referred to as "computing the global profiles over the local profiles". For example:.

The z-score represents how many standard deviations the local_distinct_actions is greater than AVG(local_distinct_actions).

In features 108A-108O described hereinabove, processor <NUM> performs the counts by counting the number of log entries <NUM> matching the specified conditions. For example, in Features 108N, the specified conditions comprise combinations of the distinct sources and the normalized application IDs.

As described hereinbelow, processor <NUM> can compute a score for each action based on the features generated for the action. In the features described hereinabove:.

<FIG> is a flow diagram that schematically illustrates a method of training model <NUM> on a set of training data, in accordance with an embodiment of the present invention. In embodiments described herein the training data comprises action log <NUM>.

In step <NUM>, processor <NUM> collects, from multiple endpoint agents <NUM> respectively executing in host computers <NUM> deployed in multiple sources <NUM>, respective sets of actions <NUM> performed, by software applications executing on the host computers, on respective raw action entities <NUM>.

In step <NUM>, using embodiments described hereinabove, processor stores information from the collected actions to respective log entries <NUM> in action log <NUM>.

In step <NUM>, processor <NUM> normalizes, in log entries <NUM>, raw entities <NUM> into normalized entities <NUM>, actions <NUM> into normalized actions <NUM>, and names <NUM> of software applications <NUM> into normalized application IDs <NUM>. As described supra, normalized actions <NUM> comprise respective combinations of action types <NUM> and normalized action entities.

Different instances of a given software application <NUM> may have different names <NUM> that reflect different builds or versions. In a software application normalization embodiment, processor <NUM> can normalize application names <NUM> by removing unnecessary information so as to determine a common normalized application ID <NUM> across all the instances. For example, processor <NUM> can normalize any of the following full application names <NUM>.

In one embodiment, if the executable file for the given software application comprises a respective file signature <NUM>, then processor <NUM> can normalize the application name of the given software application by concatenating the vendor's name (i.e., the vendor the provided or produced the given software application) to the respective normalized application ID <NUM>. In an additional embodiment, processor <NUM> can normalize the given software application by computing a hash value for the executable file <NUM> of the given software application.

In a process normalization embodiment, if a given action type <NUM> comprises process creation or process injection comprising a new process <NUM> having a given process name <NUM>, processor <NUM> can normalize the process name by removing any unnecessary information in the name. In some embodiments, if the given process comprises a respective process signature <NUM>, then processor <NUM> can normalize the process name by adding the respective process signature ID <NUM> to the respective normalized entity <NUM>.

In a domain normalization embodiment, a given action type <NUM> comprises domain access to a given domain <NUM>. In this embodiment, processor <NUM> can split each domain <NUM> into three sections. For example, the domain "a. com" can be split into:.

Since the subdomain is usually attacker-controlled, processor <NUM> can perform this normalization based on main domain concatenated with the public suffix (i.e., "site.

In an ASN normalization embodiment, a given action type <NUM> comprises an access to a given IP address <NUM>. If processor <NUM> can map the given IP address to a given ASN <NUM> comprising a geolocation, then the server processor can normalize the given IP address to the geolocation.

In a file normalization embodiment, a given action type <NUM> comprises file access of a given file <NUM>. In one file normalization embodiment, processor <NUM> can use embodiments in the software application and the process normalization embodiments described hereinabove to normalize the given file by removing any unnecessary information from the respective file name <NUM>. In other file normalization embodiments, processor <NUM> can use the respective file size <NUM> and/or the respective file type <NUM> to normalize the given file.

In an RPC normalization embodiment, a given action type <NUM> comprises an RPC call. For example, the WINDOWS™ operating system has uses WINDOWS MANAGEMENT INSTRUMENTATION™ (WMI™) queries to query system data. WMI™ queries can be used by both legitimate applications and attackers, and the queries can be divided into three sections - SELECT, FROM and WHERE. Since the WHERE section usually contain redundant information, processor <NUM> can normalize the query by only using the SELECT and the FROM sections.

For example, processor <NUM> can use this embodiment to normalize the following WMI™ query:.

In registry normalization embodiments, a given action type <NUM> comprises a registry access to a given key <NUM> having a given path. In a one registry normalization embodiment, processor <NUM> can normalize the registry access by removing any redundant and randomized information in the path.

Some groups of registry keys <NUM> groups tend to be used for malicious activity more often than others. In another registry normalization embodiment, processor <NUM> can assign respective identifiers to each of the groups (i.e., the identifiers indicating whether or not the respective groups tend to be used for malicious activity), and use the identifiers for normalization.

In a syscall normalization embodiment, if a given action type <NUM> comprises syscall, then processor <NUM> can normalize syscall <NUM> to syscall name <NUM> (i.e., without any parameters in syscall <NUM>).

Returning to the flow diagram, in step <NUM>, using embodiments described hereinabove, processor <NUM> analyzes action log <NUM> (now comprising normalized application IDs <NUM>, normalized entities <NUM> and normalized actions <NUM>) so as to compute local and global features <NUM>. In embodiments herein, action log <NUM> comprises information (e.g., host ID <NUM>, source ID <NUM>, time <NUM>, full application name <NUM>, action type <NUM>, and raw action entity <NUM>) that processor <NUM> collects from host computers <NUM>, as well as information normalized by the server processor (e.g., normalized application name <NUM>, normalized action entity <NUM>, and normalized action <NUM>).

In step <NUM>, processor <NUM> stores the computed local and global features to model <NUM>, and the method ends.

<FIG> is a flow diagram that schematically illustrates a method of using the model to classify normalized actions <NUM> in action log <NUM>, in accordance with an embodiment of the present invention.

In step <NUM>, processor <NUM> selects the first log entry <NUM> in action log <NUM>. The selected log entry comprises a given normalized application ID <NUM> and a given normalized action <NUM>.

In step <NUM>, processor <NUM> identifies a set of local and global features <NUM> that the server processor can use to analyze the given normalized action. When analyzing normalized actions <NUM> in log entries <NUM>, processor <NUM> may use:.

In step <NUM>, processor <NUM> applies the identified features to the given normalized action so as to classify the given normalized action as either benign or suspicious. In the following example, processor <NUM> can compute a score based on the identified features, and determine the classification based on the computed score. In this example, the scores range from <NUM>-<NUM> lower scores are more suspicious, and processor <NUM> can set a threshold to determine the classification (e.g., scores below <NUM> can indicate that the given normalized action is suspicious, and scores <NUM> and greater can indicate that the given normalized action is benign).

In order for the score to not be affected by insignificant factors (e.g., the number of different sources <NUM>, the number of different host computers <NUM>, and the number of different software applications <NUM>), processor <NUM> can normalize each identified feature <NUM> so as to normalize the identified features in a [<NUM>, <NUM>] range. For example, processor <NUM> can perform the following steps:.

In some embodiments, computing the score enables processor <NUM> to compare action normalized action to the identified features, so as classify the normalized action as either benign or anomalous and therefore suspicious.

In step <NUM>, if processor <NUM> classified (i.e., using embodiments described hereinabove) the given normalized action as suspicious, then in step <NUM>, the server processor can generate an alert for the instance of the software application corresponding to the given normalized application ID. For example, the alert may comprise a notification to a systems administrator to investigate the given normalized action.

In step <NUM>, processor <NUM> determines if there are any non-selected log entries <NUM> in action log <NUM>. If there are any non-selected log entries <NUM> in action log <NUM>, then in step <NUM>, processor <NUM> selects the next log entries <NUM> in the action log, and the method ends.

Returning to step <NUM>, if there are no non-selected log entries <NUM> in action log <NUM>, then the method ends.

Returning to step <NUM>, if processor <NUM> classified the given normalized action as benign, then the method continues with step <NUM>.

<FIG> is a flow diagram that schematically illustrates a method of using the model to perform real-time classification of actions <NUM>, in accordance with an embodiment of the present invention.

In step <NUM>, processor <NUM> receives, from a given endpoint agent <NUM> executing on a given host computer <NUM>, information a new action <NUM> performed on a given raw action entity <NUM> by a given software application <NUM>.

In step <NUM>, using embodiments described supra, processor <NUM> creates a new log entry <NUM>, and populates the new log entry with host ID <NUM>, source ID <NUM>, time <NUM>, full application name <NUM>, normalized application ID <NUM>, action type <NUM>, raw action entity <NUM>, normalized entity <NUM>, and normalized action <NUM>.

In step <NUM>, processor <NUM> uses embodiments described hereinabove to identify a set of local and global features <NUM> that the server processor can use to analyze the new normalized action.

In step <NUM>, processor <NUM> uses embodiments described supra for applying the identified features to the new normalized action so as to classify the new normalized action as either benign or suspicious.

In step <NUM>, if processor <NUM> classified the new normalized action as suspicious, then in step <NUM>, the server processor can generate an alert for the instance of the software application corresponding to the new normalized application ID.

In step <NUM>, processor <NUM> updates features <NUM> with the information in the new log entry, and the method ends. In some embodiments, processor <NUM> can periodically (e.g., once every <NUM> hours) update features <NUM> with the information in new log entries <NUM> (i.e., since the previous update).

Claim 1:
A method, comprising:
identifying multiple host computers (<NUM>) executing respective instances of a specific software application (<NUM>), each given instance on each given host computer (<NUM>) comprising a set of program instructions (<NUM>) loaded, by the host computer (<NUM>), from a respective storage device (<NUM>);
collecting (<NUM>), from the host computers (<NUM>), information (<NUM>) on actions (<NUM>) performed by the executing instances (<NUM>), wherein the information comprises action types (<NUM>) and entities (<NUM>);
normalizing (<NUM>), the entities (<NUM>) into normalized entities (<NUM>), and defining, for each one of the actions (<NUM>), a corresponding normalized action (<NUM>) comprising the respective action type (<NUM>) and normalized entity (<NUM>) for said action (<NUM>);
computing features (<NUM>) based on the information (<NUM>) collected from the multiple host computers (<NUM>), the normalized entities (<NUM>), and the normalized actions (<NUM>), wherein the features (<NUM>) reflect respective metrics of the actions (<NUM>) performed by the software application (<NUM>);
comparing (<NUM>), by a processor (<NUM>), the collected information (<NUM>) for a given instance to the features (<NUM>) so as to classify (<NUM>) the given instance as benign or suspicious by computing (<NUM>) a score for each action (<NUM>) performed by the given instance (<NUM>) based on the features (<NUM>) generated for the action (<NUM>) and determine a classification based on the computed scores; and
generating (<NUM>) an alert for the given instance only upon classifying the given instance as suspicious.