Patent Description:
In recent years, it has become increasingly difficult to detect anomalous network or malicious activity carried on networks. The sophistication of intrusions has increased substantially, as entities with greater resources, such as organized crime and state actors, have directed resources towards developing new modes of attacking networks.

One fairly harmful type of intrusions pertains to the situation when an outside entity takes control of a host at a given company or organization. When this happens, the host can be controlled and used as a source of attacks against other targets or as a means to exfiltrate data from within the organization. What makes this type of attack difficult to detect is that it can be carried out over time and often is mistaken for "normal" network activity until data about the attack can be analyzed over longer periods of time.

This type of attack may be implemented among criminals in the digital world, where a controlled internal host (e.g., infected computer inside an organization's network) is used in a botnet to carry out attacks on behalf of a bot master or to exfiltrate data from an internal source.

Identification of systems that are being used as relays and jump-systems to gain or delegate access to other systems is also a difficult problem due to the scale of data flowing through large networks as well as lack of visibility to what is actually being communicated between systems. In addition, many systems can communicate using unknown or modified protocols especially those designed for malicious purposes. Furthermore, the protocol on different legs of the relay can be different which makes detection even more difficult.

What is needed is a system and method to effectively and efficiently identify such scenarios where an outside entity uses a relay host to attack or gain access to other hosts within a network.

<CIT> discloses a system and method for detecting malicious relay communications. Network communications can be received and analyzed using network components as a network switch. The received traffic can be parsed into sessions. Relay metadata can be extracted from the sessions and further be used to categorize the sessions into one or more types of relay metadata behaviors. Once a significant amount of sessions are detected an alarm may be triggered and alarm data may be generated for analysis by network security administrators.

<NPL>, discloses a method for analyzing network traffic in order to identify attacks that exploit the timing behavior of TCP and other protocols and applications in low-latency anonymity networks. Mixes have been used in many anonymous communication systems and are supposed to provide countermeasures to defeat traffic analysis attacks. Flow-correlation attacks are addressed, by which an adversary attempts to analyze the network traffic and correlate the traffic of a flow over an input link with that over an output link. Two classes of correlation methods are considered, namely time-domain methods and frequency-domain methods.

The invention is defined by the independent claims, taking due account of any element which is equivalent to an element specified in the claims. The disclosed embodiments provide an approach for identifying network relays and jump-systems by extracting metadata from network traffic and deducing behavioral indicators of an actor accessing a resource. In some embodiments, network packets for a network, such as an internal network, are parsed or processed into flows of unidirectional or bidirectional communications between computing entities inside the network. Metadata that comprises information that describes the network communications without requiring the content or the packets of the network communication to be inspected may then be generated. In some embodiments, the metadata is organized into sessions that correspond to sources and destinations. In some embodiments, a time-series representation of the sessions of metadata is constructed. In some embodiments, the time-series representations of the destination group and the source group are compared with each other to compute scores that reflect the similarity between how close the two given sessions are in their network behavior. In some embodiments, an alarm will be generated if the time-series comparisons exceed a threshold.

Further details of aspects, objects, and advantages of some embodiments are described below in the detailed description, drawings, and claims. Both the foregoing general description and the following detailed description are exemplary and explanatory, and are not intended to be limiting as to the scope of the embodiments.

Various embodiments of the invention are directed to a method, system, and computer program product for identifying network relays and jump-systems by extracting metadata from network traffic and deducing behavioral indicators of an actor accessing a resource.

The drawings illustrate the design and utility of some embodiments of the present invention. It should be noted that the figures are not drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. They are not intended as an exhaustive description of how the invention or as a limitation on the scope of the invention. In addition, an illustrated embodiment need not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiment even if not so illustrated. Also, reference throughout this specification to "some embodiments" or "other embodiments" means that a particular feature, structure, material, or characteristic described in connection with the embodiments is included in at least one embodiment. Thus, the appearances of the phrase "in some embodiments" or "in other embodiments" in various places throughout this specification are not necessarily referring to the same embodiment or embodiments.

For illustrative purposes, this document will specifically describe network relays and jump-systems. However, it is noted that while this document illustratively refers to network relays and jump-systems, the invention is indeed broad enough to be applicable to other types of analysis as well. Therefore, network relays and jump-systems are only used for the sake of illustration and as an example.

One harmful type of intrusion pertains to the situation when an outside entity takes control of a host computer inside a company or organization. When this happens, the host can be controlled and used as a source for attacks against other targets or as a source to exfiltrate data from within the organization. What makes this type of attack difficult to detect is the attack could be carried out over time and often mistaken as "normal" network behavior until the data can be analyzed over a longer period of time.

This type of control is often implemented among criminals in the digital world, where the controlled internal host is used in a botnet to carry out attacks on behalf of the bot master outside of the company network or to where the controlled internal host is used as part of targeted attack to exfiltrate data from an internal source.

As noted above, when an actor uses a series of relays or jump-systems to indirectly access a target network resource, indicators of its behavior will permeate across all legs of the relay. Therefore, by measuring and quantifying these indicators from network activity and comparing them, it can be estimated which network traffic connections are controlled by a common actor. In addition, the indicators can also indicate whether the hosts are in fact part of a single long chain of connections connecting one host to another via a series of intermediate hosts.

<FIG> illustrates a time-series relay detection engine <NUM> for detecting relay attacks in an example networking environment <NUM>, as according to some embodiments. There, an internal network <NUM> comprises one or more hosts (e.g., computing entities, clients, network computers) 104a, 104b, 104c, 104d, and 104e, and a relay <NUM>, that may communicate with one another through networking devices, such as a network switch <NUM>. These hosts generally run software installed with the direct knowledge of the end user of the host or the IT (Information Technology) organization for the company that owns both the network and the host connected to it. As such, these hosts are trusted in the system.

Generally, the internal network <NUM> is isolated from other networks by network border or security devices, such as a firewall <NUM>. However, if a malicious entity (e.g., external entity attacking) <NUM> has infected one of the hosts inside the internal network, the malicious entity can use an infected host as a relay <NUM> to bypass the network firewall <NUM> and attack other computers inside the network.

For instance, the internal host, e.g., relay <NUM>, may send a first communication (denoted by a solid arrow) to the malicious entity <NUM> to indicate that the relay is ready to act as a bot for the malicious entity <NUM>. The malicious entity <NUM> may then send a communication destined for other computers in the network using the infected host as a relay. The malicious entity may send a request (denoted by a dashed arrow) destined for host 104c inside the internal network <NUM>. This communication request is first sent to the relay <NUM>. The relay <NUM> then relays, redirects, and/or reroutes the request to host 104c, which is the intended destination. Host 104c may then receive the communication, think it is simply a communication directly from another internal computer that is trusted, and send a response (denoted by the dotted arrow) back to the malicious entity <NUM> by way of the relay <NUM>.

Generally, the firewall <NUM> prevents external hosts from being able to initiate connections into the company network. However, the firewall does not usually prevent internal hosts from connecting to any other internal hosts using approved protocol such as HTTP (Hypertext Transfer Protocol), DNS (Domain Name System) etc. The firewall prevents an external attacker from directly infecting a host inside the company network - but if an external attacker can control an internal host to act on behalf of the external attacker, then the traditional firewall cannot protect the internal host from becoming a relay inside the network. In some networks, such as campus networks, the firewall may also implement NAT (Network Address Translation) functionality that hides the internal IP (Internet Protocol) address of a host when it is communicating with outside systems.

As such, despite the firewall <NUM>, the malicious entity <NUM> can use the relay <NUM> to disguise malicious communications as normal/non-malicious activity occurring between two or more computers in an internal network. If the request (e.g., dotted arrow, <FIG>) comprises a malicious script, the host 104c is more likely to run the script (and become a bot, or send private information back to the sender) because the malicious script was sent from an entity already inside the internal network <NUM>. Once host 104c is infected, the malicious entity <NUM> can hide the fact that host 104c is controlled by and in communication with malicious entity <NUM>. Embodiments of the invention provide a relay detection engine that detects these type of network attacks that use relays to bypass network security (e.g., firewall <NUM>).

The time series relay detection engine <NUM> can be implemented to detect malicious relay communication data flows on the network (e.g., relay attacks denoted by dotted and dashed lines in <FIG>). The time series relay detection engine <NUM> comprises at least a time-series vector generator <NUM> and a time-series vector comparison module <NUM>. In some embodiments, the time series generator and comparison module may not necessarily reside in the same system.

The time series relay detection engine <NUM> extracts metadata from network traffic and creates metadata sessions by identifying and organizing a source and destination for hosts in the network. A metadata time series representation of the session is created for a session. Next, the time-series vector generator <NUM> converts the metadata time series representation into a vectorized representation. This allows a similarity value to be calculated between the two pairs of hosts to determine the similarity of network behavior within a time period for the two pairs of nodes. The time-series vector comparison module <NUM> compares the similarity value between the pair of nodes <NUM>. If the similarity value passes a certain threshold value, then an alarm is generated and the set of pair of nodes may be indicated as a part of a relay system.

In some embodiments, as illustrated in <FIG>, the relay detection engine <NUM> is coupled directly to the network switch <NUM> through use of a coupler, such as TAP/SPAN. In this way, the relay detection engine may copy and/or analyze the internal network's traffic without disruption or slowing the network down. The detection system can be located in the company network in such a way that it sees the host traffic to the Internet and sees that traffic before NAT changes the internal IP addresses of each internal host's traffic.

Though the relay detection engine is illustrated as a module external to the switch <NUM>, one of ordinary skill appreciates that the engine may also be directly integrated into the switch <NUM> as a component. Further, in some embodiments, the relay detection engine <NUM> may be integrated directly into other network devices, such as a firewall <NUM> or computers/hosts inside the network (e.g., 104a, 104b, 104c, 104d, 104e, <NUM>). The relay detection engine is described in further detail in <FIG>.

<FIG> illustrates another type of network attack using intermediate servers as internal relays to access sensitive resources stored inside a partitioned area of the networking environment <NUM>, as according to some embodiments. Here, only certain hosts have access to the resources in the partitioned part of the network. As such, a malicious entity <NUM> must use the intermediate server <NUM> to access sensitive or highly confidential resources stored inside the partitioned areas inside the network. In some embodiments, the sensitive resources inside the partitioned area of the network may correspond to database servers or servers that hold financial documents. In other embodiments, the sensitive resources are any resources that may only be accessed through a certain set of hosts.

Further, as illustrated, the malicious entity <NUM> may use the intermediate server <NUM> to access sensitive resources, such as internal hosts 104c, 104d, or 104e. The intermediate server <NUM> has access to the resources in the partitioned area <NUM> for legitimate purposes. As such, if the network trusts the intermediate server <NUM>, then the malicious entity <NUM> can use the internal network's credibility/trust-permissions to access the sensitive resources. The malicious entity <NUM> may then control the intermediate server <NUM> to access the partitioned area of the network.

Across the two scenarios, the external attacker could perform a variety of tasks, examples of which include DoS (Denial-of-Service) attacks, port scanning, brute force password guessing, data exfiltration, lateral spread of infection, theft/access to sensitive resources, just to name a few. These types of malicious attacks are difficult to defend against for several reasons. Whereas host 104c would highly scrutinize or be suspicious of communications coming from external or unknown computers, host 104c may readily except or trust communications from computers in the same network (e.g., internal network <NUM>).

<FIG> illustrates a flowchart <NUM> for a high-level approach for detecting relay attacks (e.g., network attacks that involve the use of a relay host) using network metadata, as according to some embodiments. One of the key principles of this approach is that when a host uses a series of relays or jump-systems to indirectly access another host or resource, there will be a similarity in network behavior across the legs of the relay. As an example, this means that if Host A is using Host B as a relay host to reach Host C, then the network communication exchanged between Host A and Host B will be similar to the network communication exchanged between Host B and Host C.

Metadata is first received from the network. The nodes are then organized into sets of pairs, where each pair of nodes share a common node (e.g., Nodes A and B and Nodes B and C both share communication with Node B). A time-series representation of metadata of each pair is constructed and then converted into a vector representation so that a similarity value may be calculated between the vectorized time-series representations. If the similarity value between the pair of nodes passes a certain threshold value, then an alarm is generated and the set of pair of nodes may be a part of a relay system (e.g., the common node that is shared between the pair of nodes is a potential relay).

At <NUM>, metadata will be extracted from the network. The traffic from the network may be received for analysis. In some embodiments (as described above), the network traffic may be received by tapping a network device such as a network switch. All communication exchanged between hosts are captured by parsing modules and distilled into metadata. This metadata may, in some embodiments, comprise information that describes the network communications without requiring the content or the packets of the network communication to be inspected. Further, one or more metadata may be extracted from the network communications and used for further analysis. This metadata is generated periodically for each ongoing network session between two hosts.

At <NUM>, the metadata is organized into sessions that correspond to a pair of nodes which share a common node. The sessions are organized into groups of likely candidates for sources and likely candidates for destinations by looking at the source and destination of the communications within the sessions.

At <NUM>, a time-series representation of the metadata is constructed for the sessions. The time-series representation maps the metadata corresponding to network behavior between a pair of nodes such that pairs of time-series representations may be aligned together for comparison. In some embodiments, the time-series representation may be mapped onto corresponding locations along a first axis with a height along a second axis. As an example, the first axis may correspond to time and the second axis may correspond to network attributes such as bytes sent and/or bytes received. The time-series representations are then converted into vectorized time-series representations of sessions. The details of creating the time-series representation will be further described below in <FIG> and <FIG>.

At <NUM>, a comparison is made between the vectorized time-series from the destination groups with the time series from the source groups. The comparison is made to find similarities/differences between vector representations of sessions. In some embodiments, vector comparison metrics such as cosine similarity is used to quantify how close two given sessions are in their actor-behavior space. In some embodiments, a similarity value is computed which represents the similarity between two given sessions.

In some embodiments, it may be possible that due to system delays and other extraneous errors, the time-series representation of two traffic streams may not be perfectly aligned. This may lead to errors when comparing them to each other. The system recognizes these irregularities and variations as jitters and accounts for them by performing a series of comparisons between two time-series at various small temporal offsets and then selecting the best results from the comparisons. In some embodiments, the identification of threats can involve further higher order factors such as persistent observation of the relay activity. A process of comparing shifted representations between two time-series representations of metadata will be further described in <FIG> and <FIG>.

At <NUM>, if the similarity value indicates that the network communication is likely to be part of a relay attack, alarms may be generated to alert network administrators of the potential malicious activity. In some embodiments, the alerts are generated when the similarity value exceeds a threshold.

In some embodiments, certain nodes and connections may be prioritized for processing. Extraneous indicators for prioritizing nodes and connections may include nodes that are likely to be problematic or nodes that correspond to high value targets (e.g., confidential information). These indicators can include other types of detected threats, such as specific behaviors known to precede establishing of relays, targets and sources of previously detected relays (e.g., to look for the entire chain).

<FIG> illustrates internal aspects of the relay detection engine, as according to some embodiments. At <NUM>, the relay detection engine <NUM> may receive traffic for the internal network <NUM> through the above-disclosed means, such as through a network switch TAP. A flow pre-processor module <NUM> may use a parsing module or unit <NUM> that can analyze the network traffic and divide or break it up into unidirectional flows (e.g., outgoing, incoming) and session datasets. In some embodiments, the parsing module utilizes packet information, such as source and destination address to parse the network traffic into the flows.

An extraction module <NUM> extracts network metadata that describes the network traffic or flows. In some embodiments, the metadata may comprise the amount of bytes in a flow, the first N bytes of a flow (e.g., the first <NUM> bytes of a flow), the source address, the destination address, variations in flow/packet size, or other statistical data that may be used as metadata to describes the flows without requiring deep content inspection of the packets.

An analyzer module <NUM> may employ a categorization unit <NUM> that uses the metadata to categorize the flows into one or more known network attribute criteria and time-series vector generator <NUM> to create quantitative abstractions of the metadata that capture actor behavior. The analyzer module <NUM> may comprise at least a categorization unit <NUM>, a time-series vector generator <NUM>, and a vector comparison module <NUM>.

In some embodiments, the categorization unit <NUM> uses the incoming-flow metadata and outgoing flow metadata to categorize the flows into one or more behavior groups. In some embodiments, the incoming flow metadata and the outgoing-flow metadata are combined to generate global session data. The global session data may then also be used to categorize the network traffic into one or more behavior groups, as further described below.

In some embodiments, the time-series vector generator <NUM> converts the time-series representations of metadata into a fixed length vector. The metadata is projected into a vector where it can be compared by the vector comparison module <NUM> to find similar projections from other sessions. Further details about how to create the vectorized representations are described in further detail in <FIG> and <FIG>.

A vector comparison module <NUM> finds similarities between vector representation of sessions by calculating a similarity value between the vectorized representation of metadata sessions. The vector comparison module may use any suitable vector comparison metric such as cosine similarity to calculate a comparison value between vector representations. The comparison values can be used to determine which network traffic connections are being controlled by a common actor and may in fact be part of a chain of connections connecting one host to another via a series of intermediate hosts.

A score module <NUM> may use a threshold unit <NUM> that may retrieve past historical data from one or more storage devices and determine whether one or more sessions are likely to be part of a network attack using relays. In some embodiments, the historical data is retrieved and a comparison may be performed between the most recent session categorization (e.g., for a given pair of hosts) and past historical activity. From comparison, if it is determined that the behavior is equal to or beyond a relay threshold limit, then an alarm (e.g., alarm data, reporting data, output data) may be generated and output at <NUM>. In some embodiments, when a session is categorized as a type recognized relay behavior, a data structure (e.g., reference counter) may be updated that corresponds to the involved hosts. If the reference counter is equal to or surpasses a threshold limit, then an alarm (e.g., alarm data, reporting data) may be generated and output at <NUM>.

<FIG> shows a flow for an approach for constructing time-series representations of metadata. The time-series representations of metadata allow sessions to be measured and compared to other sessions.

At <NUM>, network metadata is identified at particular points in time. The metadata may be received for analysis from the network traffic. In some embodiments (as described above), the network traffic may be received by tapping a network switch.

Next, at <NUM>, the network metadata is mapped at a corresponding location along a first axis and a second axis. The first axis corresponds to time periods (e.g., a minute) that comprises of smaller time points (also known as bins of time) that represent a small interval of time (e.g., <NUM> second). The second axis corresponds to values that correspond to different network attributes. For example, network attributes may correspond to criteria such as the number of events per time period, the pattern of distribution of events, the bytes sent or received, and/or a relative/ratio between bytes sent and received.

At <NUM>, a first time point within a time period along the first axis of the plot is identified. At <NUM>, a determination is made if the time point contains an exchange of packets and data. If there was no exchange of data in the time point, then the value in the vector, for this time point, is set to <NUM> at <NUM>. However, if there was an exchange of packets and data for the time period, then, at <NUM>, a further determination is made whether the previous time point corresponds to a network attribute value (e.g., the value of the vector is not <NUM>). If the previous time point does not contain a value, then plot the point along a y axis that corresponds to the network criteria, at <NUM>. If the previous time point does contain a value, then set the vector value for that time point to <NUM>, at <NUM>. If step <NUM> is being executed for an initial time point within the time interval (e.g., there is no previous time point), then set the vector value for that time point to the network attribute value.

At <NUM>, a determination is made if there are any additional intervals along the first axis that have not been analyzed to see if the interval contained an exchange of packets and data. If it is determined that there are no more additional intervals, then the method is finished. However, if it is determined that there are additional intervals that have not been analyzed then the flow chart loops back to <NUM> and a determination is made if the time point contains an exchange of packets and data.

<FIG> illustrates an example of a creation of a time series representation of metadata.

In <FIG>, the system identifies combinations of sources and destinations for a node. In this example, Node B is a common node between communications for Node A to Node B and Node B to Node C. As previously explained, the detection system and method is based at least in part on a key principle that when an actor uses a series of relays or jump-systems to indirectly access a target network resource, network behavior between a pair of nodes will permeate across all legs of the relay. In this example, the communications between Node A to Node B (<NUM>) and Node B to Node C (<NUM>) will be measured and organized around Node B to determine the similarities in network behavior over a period of time between the pairs of nodes.

In this particular example, the communication (e.g., <NUM>, and <NUM>) exchanged between hosts (e.g., Node A, Node B, and Node C) are captured by packet analyzers <NUM> and distilled into metadata. The distillation is performed at a predetermined resolution by aggregating packet statistics into a series of time bins. These aggregate statistics include number of packets exchanged between the host at that time point, sum of bytes exchanged in that time point, and many other stats. In some embodiments, the time point intervals may be less than a second.

As depicted in <FIG>, once the sources (e.g., Node A to Node B) and destinations (Node B to Node C) relative to Node B are identified, a network security device such as a packet analyzer <NUM> may be employed to organize and capture all network packets between Node A to Node B <NUM> and <NUM> and from Node B to Node C <NUM> and <NUM>. The packet analyzer may access and create a copy of the network communications by performing a packet capture and saving metadata corresponding to the network communications in one or more packet capture files to the database <NUM>. When the network packets are received, the network communications may be parsed into flows that correspond to sessions (e.g., <NUM> and <NUM>). In some embodiments, the packet analyzer corresponds to a network sniffer.

<FIG> shows the system adding a new layer of abstraction onto the metadata by creating time-series representations of the metadata for communications between Nodes A to B and Nodes B to C. Here, the metadata corresponding to network traffic between Node A and B <NUM> and the metadata corresponding to network traffic data between Node B and C <NUM> are mapped onto corresponding locations along a first axis with a height along a second axis. In some embodiments, the first axis corresponds to time and the second axis corresponds to network attributes. In the example in <FIG>, the network attributes corresponds to bytes received and bytes sent. Therefore, the mapped points of Node A and B <NUM> indicate bursts of activity (e.g., bytes sent from A to B or bytes received from A to B) and periods of inactivity between Nodes A and B. If Node B is indeed being controlled by Node A to reach Node C, then the same bursts of activity (e.g., bytes sent from B to C or bytes received from B to C) and same periods of inactivity between nodes B and C should be reflected in the network behavior over a time period between Node B and C <NUM>.

<FIG> shows selecting a specific time window from the plot for analysis. In this example, the time interval <NUM> may be a minute long. The time interval <NUM> is made up of smaller time points that correspond to bins of time. Here, the time interval <NUM> corresponds to the first time period between an exchange of packets between the source (e.g., Nodes A and B) is selected. Accordingly, a similar time interval <NUM> that correlates to the destination (e.g., Nodes B and C) is also selected for analysis.

Fig. 6E-J shows an example of converting the time-series representation of the specific time window. The time-series representation of 803a corresponds to the data exchanged between Nodes A and B, whereas the time-series representation of 807a corresponds to the data exchanged between Nodes B and C.

In one embodiment and for illustrative purposes only, a value of <NUM> will be entered when there is any indication of data exchanged and a value of <NUM> will be entered when there is no indication of data exchanged. Alternatively, other vector values besides <NUM> and <NUM> may be used, particularly, using a value (e.g., bytes sent or bytes received) that corresponds to network data behavior instead of <NUM>.

<FIG> depicts identifying an initial time point within the time interval along the first axis. Here, a determination is made that there are no packets or data that is exchanged at the initial time point in both 803a and 807a. As such, the vector value for the time points in 803a and 807a is set to <NUM>. A determination is then made as to whether there are additional time points within the interval along the first axis. Here, there are additional time points within the interval along the first axis.

<FIG> depicts identifying the next time point within the time period. In 803a, there is an exchange of packets and data, so a determination is made as to whether the previous point contained a value. Here, the previous time point (e.g., the initial time point in <FIG>) did not contain a value so the vector value for the current time point is set to <NUM>. In 807a, there are no packets or data that is exchanged at this time point so the value of the time point is set to <NUM>. Then, as explained above, it is determined that there are additional time points within the interval along the first axis.

<FIG> depicts identifying the next time point within the time period. In 803a, packets and data are exchanged at this time point, so a determination is made as to whether the previous point contained a value. Here, the previous time point (e.g., the time point in <FIG>) contains a value. This means that the vector value for this time point is set to <NUM> despite the fact that there are packets and data exchanged at this time point. In some embodiments, a value other than <NUM> may set when there are packets and data exchanged at this time point. In 807a, the vector value for this time point is set to <NUM> because there are packets and data exchanged at this time point and the previous time point (e.g., the time point in <FIG>) did not contain a value. Again, a determination is made as to whether there are additional time points left in the time period to analyze.

<FIG> depicts identifying the next time point within the time period. In 803a and 807a, both time points indicate that data is exchanged. However, a value of <NUM> is set for this time point because the previous time point (e.g., time point in <FIG>) contained a value. The next time point is analyzed because there are additional time points within the interval.

<FIG> depicts identifying the next time point within the time period. In 803a, the time point contains a new exchange of packets and data so there is an additional check to determine whether the previous time point (e.g., time point in <FIG>) contained a value. Here, the previous time point did not contain a value so the vector value of the time point is set to <NUM>. In 807a, there is an exchange of packets and data, but the time point for this value is set to <NUM> because previous time point (e.g., time point in <FIG>) contained a value. The vectorized representation is completed because there are no more additional time points within the interval along the first axis.

As previously explained, for illustrative purposes only, a vector values of <NUM> or <NUM> are assigned to the time point depending on data and packets. However, it should be clear that the vector values are not necessarily limited to values of <NUM> and <NUM>. In some embodiments, the vector values may be set to any scalar value or vector of values that correspond to network attributes.

In some embodiments, the network attributes that are measured on the second axis may correspond to network measurements such as density (e.g., number of events per time period), a pattern distribution of events, size of bytes sent and received, or relative ratio between the bytes sent and received. The different network attributes may be calculated for the same session and may be normalized and combined together to create an overall score. This process is further explained in further detail in <FIG>.

<FIG> shows a flow for an approach for comparing time-series representations of the destination group to the source groups.

At <NUM>, a particular time interval is selected to determine the similarities between the representations of a source metadata vector (e.g., first leg of communication) and a corresponding destination metadata vector (e.g., second leg of communication). The source metadata vector and the destination metadata vector are first aligned according to the time intervals to determine a similarity value between the two vectors at <NUM>. At <NUM>, vector comparison metrics such as cosine similarity may be used to quantify how similar the two given sessions are in their network behavior. Any suitable vector comparison method may be utilized.

In some embodiments, the time-series representations may not be perfectly aligned, which may lead to errors when comparing them with each other. As such, at <NUM>, the system recognizes these irregularities and variations as jitters and accounts for them by shifting the destination metadata vector to better align the two sessions. When the two vectorized sessions do not perfectly align, the destination metadata will be shifted one time point in either direction. In some embodiments, the source metadata vector may be shifted instead of the destination metadata vector for alignment purposes.

At <NUM>, a comparison value (e.g., similarity value and/or difference value) is calculated between the source metadata vector and the shifted destination metadata vector. At <NUM>, a determination is made to see whether there are additional shifts left for the destination metadata vector to be shifted to. By performing a series of comparisons between the source metadata vector and the destination metadata vector, the best result can then be selected from the shifted alignments, at <NUM>.

<FIG> illustrates an example of a comparison of a shifted time series representations of network metadata. <FIG> shows different shifted alignments (e.g., <NUM>, <NUM>, and <NUM>) between the network behavior between a representation of network behavior between Nodes A to B and a shifted representation of network behavior between Nodes B to C. A similarity value may be calculated from the each shifted comparison, where each shifted comparison corresponds to a source metadata vector that is stationary and a destination metadata vector that is shifted at least a time point. The similarity value corresponds to the similarity in match between a source metadata vector and a destination metadata vector. In some embodiments, an individual score may be further calculated from the similarly value.

In other embodiments, a phase-locked loop may be implemented to compare the vector representations. A phase-locked loop may be useful to use when large networks of computers are involved such that a comparison and computation of similarity values is computationally expensive.

In some embodiments, an overall score is calculated by aggregating normalized individual scores corresponding to different matches between the pair of nodes within a single time period. An overall score reflects whether there is persistence of match activity over a period of time. If the overall score (e.g., persistence of match) is beyond a threshold, then an alert is generated. In some embodiments, the alert is based at least on the frequency of the matches between particular nodes within a time interval. In some embodiments, the overall score can also take into account the irregularities and jitters.

In some embodiments, a correlation scheme may be implemented to apply weightings for a scoring system. The individual scores correspond to scores for the same time-series representation but with different network criteria applied. In some embodiments, these individual scores may be normalized before they can be compared to each other. The weights for particular criteria are applied to the individual score(s). As an example, the pattern of distribution of events score may be more heavily weighted than the number of events per time period criteria. Then, the value (e.g., numerical value) of the weighting is aggregated. If the aggregated weighted scores surpass a threshold, then, a global signal indicating a positively identified relay or jump-system communication and/or alert data is generated. However, if the threshold is not surpassed then the network data (e.g., sessions) is ignored.

In some embodiments, the identification of threat can involve further higher order factors like persistent observation of the relay activity. Hosts and connections may be prioritized for processing based at least on using extraneous indicators of threat. When large networks of computers are involved the disclosed invention will monitor extraneous indicators of threat and accordingly prioritize hosts and connections for processing. In some embodiments, the extraneous indicators can include other types of detected threats, specific behaviors known to precede establishing of relays, targets, and sources of previously detected relays.

In addition, the system can look for chains of relays by comparing relays detected on multiple hosts to see if destinations or destination connections of one host was flagged as a source or source connection of another host. The scores and alerts between pairs of nodes may be saved into a database. This allows detection of entire chains of relay communication.

<FIG> is a block diagram of an illustrative computing system <NUM> suitable for implementing an embodiment of the present invention for performing intrusion detection. Computer system <NUM> includes a bus <NUM> or other communication mechanism for communicating information, which interconnects subsystems and devices, such as processor <NUM>, system memory <NUM> (e.g., RAM), static storage device <NUM> (e.g., ROM), disk drive <NUM> (e.g., magnetic or optical), communication interface <NUM> (e.g., modem or Ethernet card), display <NUM> (e.g., CRT or LCD), input device <NUM> (e.g., keyboard), and cursor control. A database <NUM> may be accessed in a storage medium using a data interface <NUM>.

According to one embodiment of the invention, computer system <NUM> performs specific operations by processor <NUM> executing one or more sequences of one or more instructions contained in system memory <NUM>. Such instructions may be read into system memory <NUM> from another computer readable/usable medium, such as static storage device <NUM> or disk drive <NUM>. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and/or software. In one embodiment, the term "logic" shall mean any combination of software or hardware that is used to implement all or part of the invention.

The term "computer readable medium" or "computer usable medium" as used herein refers to any medium that participates in providing instructions to processor <NUM> for execution. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as disk drive <NUM>. Volatile media includes dynamic memory, such as system memory <NUM>.

Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read.

In an embodiment of the invention, execution of the sequences of instructions to practice the invention is performed by a single computer system <NUM>. According to other embodiments of the invention, two or more computer systems <NUM> coupled by communication link <NUM> (e.g., LAN, PTSN, or wireless network) may perform the sequence of instructions required to practice the invention in coordination with one another.

Computer system <NUM> may transmit and receive messages, data, and instructions, including program, i.e., application code, through communication link <NUM> and communication interface <NUM>. Received program code may be executed by processor <NUM> as it is received, and/or stored in disk drive <NUM>, or other non-volatile storage for later execution.

Claim 1:
A method (<NUM>) for identifying malicious network communications comprising:
receiving network traffic by tapping at least one network device (<NUM>) of a network (<NUM>);
extracting (<NUM>) a set of metadata from the network traffic, the set of metadata corresponding to network traffic between nodes (<NUM>, <NUM>) of a plurality of pairs of nodes (<NUM>, <NUM>; <NUM>, 104c), the set of metadata describing network communications;
generating (<NUM>) time-series vectors representing the set of metadata, each time-series vector of the time-series vectors corresponding to a pair of nodes (<NUM>, <NUM>) of the plurality of pairs of nodes (<NUM>, <NUM>; <NUM>, 104c);
computing, between respective pairs of nodes (<NUM>, <NUM>; <NUM>, 104c) that have one node (<NUM>) in common, a plurality of similarity values based on corresponding time-series vectors by:
performing a plurality of comparisons of a source metadata vector of the time-series vectors corresponding to a first pair of nodes and a destination metadata vector of the time-series vectors corresponding to a second pair of nodes to generate corresponding similarity values for the pair of nodes, wherein:
a second comparison of the plurality of comparisons occurs after the source metadata vector or the destination metadata vector undergoes an adjustment using a temporal offset, and
a first comparison of the plurality of comparisons occurs before the source metadata vector or the destination metadata vector undergoes the adjustment using the temporal offset; and
generating (<NUM>) an alarm when a similarity value of the plurality of similarity values are beyond a threshold indicating that at least one pair of nodes (<NUM>, <NUM>) of the plurality of pairs of nodes (<NUM>, <NUM>; <NUM>, 104c) may be part of a relay arrangement.