Network security monitoring and correlation system and method of using same

A network security monitoring and correlation system for providing a three-dimensional visualization of network traffic overlaid with security alerts and other relevant discrete data. The system may comprise an application server communicably linked to a client. The server functions to retrieve network traffic metadata and relevant discrete data associated with individual computer hosts and connections in the monitored network, process the network traffic data by building a graph data structure, and then embedding within the graph data structure one or more layers of additional information about the individual computer hosts and connections derived from the discrete data. The client functions to produce a three-dimensional visualization of the network environment by parsing the graph data structure received from the server and then spawning computer hosts and connections in the 3-D environment. The client will then add the overlay information to the appropriate hosts or connections, with the overlay information preferably being represented within the 3-D environment as a particular color, shape, size, position, or a changing dynamic value.

BACKGROUND

Networked computers connected via the Internet are subject to intrusion attempts on a regular basis. Due to the exponential increase in network connections, intrusion attempts via automated attack tools, and other network events that share exponential growth characteristics as computer networks grow in size and complexity, current network monitoring systems and methods are limited in their ability to present enough information concurrently to human analysts. This limiting factor in computer security analysis results in several negative performance impacts.

First, existing security monitoring systems typically present information to analysts in two-dimensional data visualizations, primarily in spreadsheet format. Due to the vast number of alerts and data present in any network of considerable size, and the relatively small display area available to present information, analysts can only see a very small amount of data at a time when performing a security analysis.

Second, existing security monitoring systems are significantly limited in presenting a correlation of security event data between multiple systems and over periods of time. Analysts are required to mentally visualize multiple events in order to properly assess the larger security state. For example, security threats can oftentimes come in the form of unauthorized access of multiple computers by multiple users. However, due to the limitations of existing security monitoring systems, the onus is on the human analyst to visualize the security event and understand the environment and context within which it occurred.

Third, in order to properly prioritize and respond to computer security events, human analysts must have both experience in information security and knowledge of the computer network environment. As intrusion attempts and the complexity of networks escalate, the job qualifications of security analysts correspondingly increase, resulting in a shortage of qualified analysts in the workforce.

SUMMARY OF THE INVENTION

The network security monitoring and correlation system of the present invention utilizes mixed reality techniques to display contextual and prioritized data visualization schemes, thereby allowing a human analyst to quickly inspect and understand the environment surrounding computer hosts within a monitored network. In a preferred embodiment, the system is designed to provide contextual and intuitive visualization of security data in order to prioritize security response activity. By providing a three-dimensional visualization of network traffic overlaid with security alerts and other relevant discrete data, the system can display exponentially more usable data to human analysts than prior art systems. The system effectively visualizes the network environment for the analyst, thereby providing the proper context of security events so that the analyst can develop an appropriate response for addressing any intrusion(s).

A network security monitoring and correlation system embodying features of the present invention may comprise an application server communicably linked to a client. The server functions to retrieve network traffic metadata and relevant discrete data. The server will process the data by building a graph data structure using the network traffic metadata and then embedding within the graph data structure one or more layers of additional information derived from the network metadata and the discrete data about individual computer hosts and connections active within the monitored network. In a preferred embodiment, the application server may comprise a retrieval engine module for retrieving the collected network traffic metadata and relevant discrete data, a graph generator module for processing the retrieved network metadata and building the graph data structure, and an overlay generator module for cooperating with the graph generator to embed the additional information into the graph data structure.

The client functions to produce a three-dimensional visualization of the network environment by parsing the graph data structure received from the server. The client will spawn computer hosts and connections in the 3-D environment using the network traffic metadata and relevant discrete data organized within the graph data structure, assigning a three-dimensional object to both the hosts and connections. The client will then add the overlay information to the appropriate hosts or connections, with the overlay information preferably being represented within the 3-D environment as a particular color, shape, size, position, or a changing dynamic value. The network traffic and overlay information can then be visualized in the three-dimensional environment. In a preferred embodiment, the client can comprise a network worker module for handling communications and the transfer of data between the client and the server, a 3-D object generator module for creating the three-dimensional environment, a communicator module for spawning nodes and connections within the three-dimensional environment, movement and layout modules, and a mixed reality graphical user interface.

The above summary is not intended to describe each illustrated embodiment or every possible implementation. These and other features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings.

DETAILED DESCRIPTION

A network security monitoring and correlation system utilizing mixed reality techniques is described herein. Detailed embodiments of the present invention are disclosed; however, it is to be understood that the disclosed embodiments are merely exemplary of the invention, which can be embodied in various forms. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present invention in virtually any appropriately detailed environment. Alternate embodiments may be devised without departing from the spirit or the scope of the invention. Further, the terms and phrases used herein are not intended to be limiting; but rather, to provide an understandable description of the invention. While the specification concludes with claims defining the features of the invention that are regarded as novel, it is believed that the invention will be better understood from a consideration of the following description in conjunction with the drawing figures, in which like reference numerals are carried forward.

The network security monitoring and correlation system of the present invention utilizes mixed reality techniques to present a three-dimensional depiction of the network activity that can be overlaid with security alerts and/or other relevant discrete data (sometimes referred to herein as “discrete overlay data”). The discrete overlay data includes any data useful for being superimposed over the network traffic. In certain embodiments, the discrete data may include alert data, user access data, endpoint data, and threat intelligence data. The system allows a human analyst to quickly inspect and understand the environment of the monitored network. The term mixed reality, as used herein, is meant to encompass the use of virtual reality visualization systems and interfaces, augmented reality visualization systems and interfaces, and real visual display systems and interfaces, such as monitors and keyboards.

The system can present each type of network traffic data as a particular three-dimensional object—such as spheres, cubes, pipes, or other 3-D objects. Attributes of the network traffic data can be distinguished in the 3-D environment by a particular color, size, position, or changing dynamic values of these three-dimensional objects. Meanwhile, network security alerts and other relevant discrete data can be represented within the 3-D environment as a particular color, shape, size, position, or a changing dynamic value (e.g., flash, pulse, spin, etc.).

FIGS. 1a-1gdepict various schemes that can be utilized by the system1for displaying the network environment overlaid with additional information derived from data collectors such as intrusion detection systems (IDS's), log aggregators, antivirus applications, and cyber-threat intelligence feeds. For example,FIG. 1ais an illustration of an exemplary “sandwich” or “layered” 3-D mixed reality environment generated by the system of the present invention. The private nodes7(i.e., those nodes in the private network) are positioned in a first section10, the public nodes8(i.e., those nodes in the public network) are positioned in a second section11disposed spatially above the first section10, and the external nodes9(i.e., those nodes in the external network) are positioned in a third section12disposed spatially above the second section11. Connections6between the nodes can be displayed as lines or pipes having variable dimensions proportional to the volume of data.

By way of another example,FIG. 1bis an illustration of an exemplary “onion” 3-D mixed reality environment generated by the system of the present invention. The onion layout consists of three concentric spheres, with the private nodes7being positioned in the inner sphere10, the public nodes8being positioned in the middle sphere11, and the external nodes9being positioned in the outer sphere12.

FIGS. 1e-1gdepict a preferred embodiment of the onion layout rendered in a 3-D environment, whereby a map of the world is projected onto an interior surface of a three-dimensional sphere. The analyst's default position for analyzing the network is within the sphere. A 3-D peripheral device, such as a virtual reality head-mounted display, is ideally suited for navigating the 3-D environment. The network traffic is displayed within the sphere as follows. The private nodes7—which represent computer hosts in the monitored network—are preferably clustered proximate to the center of the sphere. The analyst, whose default position within the 3-D environment is also at the center of the sphere, will perceive these private nodes as being proximate to the analyst. External nodes9are positioned on the map of the world according to their geolocation, with their geolocation being derived from the network traffic metadata. Meanwhile, public nodes8are spatially positioned between the center of the sphere and the sphere's interior surface. Connectivity between the nodes are preferably displayed as pipes6having variable dimensions proportional to the volume of data, with superimposed arrows indicating the flow of data between the private, public, and external hosts7,8,9. The analyst, when at its default position, will perceive the public nodes and external nodes as being more distal to the analyst as compared to the private nodes. Thus, if an analyst wishes to trace a connection from a private node to an external node geolocated on the sphere's interior surface, the analyst will experience the sensation of traveling through space towards the projected map.

Referring now toFIG. 1d, a private node7ais shown as having a security alert associated with it. The system of the present invention allows an analyst to visually inspect the environment surrounding the computer host(s) associated with a security alert and quickly take the appropriate remedial action. For example, as shown inFIGS. 1eand 1f, the system allows an analyst to trace the private node's connections to external nodes9that are geolocated on the projected map of the earth. In instances where numerous external nodes are active in the monitored network, the system1will condense these nodes into a singular point on the projected map to declutter the 3D environment (seeFIG. 1e). However, when an analyst desires to visually inspect the connections to specific external nodes associated with the security alert, the system1allows an analyst to explode the condensed nodes, thereby showing each external node9at its specific geolocation on the projected map (seeFIG. 1f).

In addition to visually examining the network environment surrounding security alerts, the system1can also provide a listing of alerts existing within the monitored network environment. InFIG. 1g, an embodiment of an alert details interface20is shown. The alert details interface20can include a source host identifier list21and a destination host identifier list22consisting of the source host IP address and the destination host IP address for each computer host associated with each alert.

FIG. 2ais a block diagram illustrating the environment in which an embodiment of system1of the present invention can operate. In a preferred embodiment, the system1works as a server-client model to allow for scalability. However, in alternative embodiments, the system1could be employed on a single workstation, or it could be provided as a cloud-based solution whereby the system components and routines are accessed via the Internet by an analyst using a personal computer, mobile computing device, etc.

Referring toFIG. 2a, the network security monitoring and correlation system1can comprise a one or more data collectors101, a system database102, an application server200, and a client400.

The data collectors101function to collect network traffic metadata and discrete data. The network metadata is used for creating a three-dimensional visualization of network traffic by the system1. The discrete data includes specific, descriptive data about the content (the hosts and connections) of the network metadata activity used to generate the 3-D environment. The discrete data can be separated into various visualization layers by the system1. In certain embodiments, discrete data layers may include cybersecurity alert data, user access data, endpoint data, and threat intelligence data, and any other discrete data layers available in the system database102. A variety of data collectors101or sensors—such as such as network traffic collectors, intrusion detection systems (IDS's), log aggregators, antivirus applications, and cyber-threat intelligence feeds—can be employed for continuous, real-time collection of the relevant metadata and discrete overlay data. In certain embodiments, the data collectors101can be executed on separate network hosts on the administrative network. In other embodiments, the data collectors101can be integrated into the application server200.

The system database102stores the metadata and discrete data collected by the data collectors101. In a preferred embodiment, the system database102is a non-relational (NoSQL) database, with all data preferably being stored in the system database102as documents—a data storage format whereby the data is stored as a collection of key-value pairs that can be queried, and subdivided into listings of types, based on the contents. The data collected by the system1is generally divided into two classes: network traffic metadata and discrete data. Each network traffic metadata document will contain metadata regarding a connection between two computer hosts, while each discrete data document will contain specific data that is either actual content—or describes actual content—about a particular event (e.g., alerts, user access, etc.) that occurred in relation to a host or connection within the query set timeframe. In other embodiments, relational databases or other database formats may be utilized. A single system database102can be utilized with the system1, or alternatively, multiple system databases102may be employed. The system database102can be a separate host on the administrative network, or alternatively, the system database102can be integrated into the application server200.

The server200retrieves unstructured data from the system database102within a specified time interval through a set of database queries. The server200then processes this unstructured data and produces a snapshot—a graph data structure that contains the network metadata and discrete data necessary for populating and visualizing the network environment. The snapshot graph structure is utilized by the client400to generate and populate the 3-D environment.

FIG. 2billustrates the composition of a snapshot30data structure. The snapshot30is comprised of network traffic metadata and discrete data collected during the snapshot timeframe, which is the time period defined by the start date time35and the snapshot end date time36. The snapshot30is organized in a graph data structure which contains a vertices list31and an edge list32. Each vertex60in the vertices list31represents an individual computer host that was recorded in the network traffic metadata during the snapshot timeframe. Meanwhile, each edge70in the edge list32represents a connection between two computer hosts (i.e., vertices) that was recorded in the network traffic metadata during the snapshot timeframe. The snapshot30further comprises a sensor name index list33, which is a listing of all sensors which have identified the computer hosts and connections in the network traffic. A token37can also be utilized as a mechanism for authentication before transmitting the snapshot to a particular client400.

In order to build an edge in the graph data structure, the system1may utilize normalized fields40—metadata types derived from the network traffic metadata documents present in the system database102for the snapshot timeframe and which are required to accurately portray each network connection in the 3-D environment. The normalized fields40include the client name index41; the destination host identifier42(e.g., the destination IP address); the destination host port43, the source host identifier44(e.g., the source host IP address); the source host port45; the sensor name identifier46; and the start datetime35. In order to build a vertex in the graph data structure, the system1will utilize metadata types derived from the network traffic metadata documents which are required to accurately portray each computer host in the 3-D environment, namely the client name index41, the IP address62, the IPv6 flag63(i.e., which identifies the internet protocol version of the IP address), and the sensor name identifier46(i.e., the sensor which collected the data). An exemplary network traffic metadata document is shown inFIG. 2c.

Once the graph data structure is generated, the system1will overlay each vertex and each connection in the graph data structure with one or more layers of network traffic additional information54and, if available, with one or more layers of discrete data additional information55. The network traffic additional information54consists of non-normalized network traffic metadata derived from the network data documents and organized within the overlay information map50. The discrete data additional information55consists of additional information (e.g., intrusion alerts) about computer hosts and connections derived from the discrete data and organized within the overlay information map50. An exemplary IDS alert discrete data document is shown inFIG. 2d.

The overlay information map50is used to organize the additional information54,55before it is embedded into the graph data structure. In a preferred embodiment, the discrete data and non-normalized traffic metadata is organized within the overlay information map50by a group ID51(i.e., an integer assigned to a particular overlay data layer); a group name52; and document type name53. For example, in a preferred embodiment, the scheme221is configured to assign the following group IDs51, group names52, and type names54to the following additional information documents:

An exemplary process of creating the graph data structure and overlaying the network traffic additional information54is shown inFIG. 6a. An exemplary process of overlaying the discrete data additional information55is shown inFIG. 6b.

The depicted metadata and discrete data types inFIG. 2bthroughFIG. 2dare not exhaustive. One skilled in the art will readily recognize that a wide variety of data types may be collected and utilized by the system1of the present invention, depending on the types of discrete data that one desires to superimpose over the three-dimensional visualization of metadata. Moreover, whileFIG. 6aandFIG. 6bshow the network traffic additional information54being embedded into the graph data structure prior to the discrete data additional information55, one skilled in the art will appreciate that the order of embedding the additional data into the graph data structure can vary in alternative embodiments of the present invention.

Referring again toFIG. 2a, the application server200functions to retrieve the collected network metadata and discrete data, process the collected data to generate a graph data structure, and then transmit the data to the application client400. The application server200can feature a retrieval engine210, a graph generator230, an overlay generator240, and a client handler280.

The retrieval engine210retrieves network traffic metadata and discrete data from the system database102and provides this data to the graph generator230and the overlay generator240. The retrieval engine210is configured via the data types information schema221, which allows the server200to be configured and re-configured as different data collectors101are added to the system1without having to re-compile the server application. The data types information schema221enables the retrieval engine210to collect different document formats from the system database102by specifying document formats by type (e.g., network traffic documents, alert documents, etc.), and the schema221also enables the retrieval engine210to parse the unstructured data retrieved from the system database102using standard JSON encoding and decoding processes. Meanwhile, the data types information schema221enables the graph generator230to organize the network traffic metadata and discrete data into a graph data structure by defining the normalized fields40for each data type (seeFIG. 2b) and by defining the groups (i.e., the group ID51and the group name52) for the overlay information map50.

The graph generator230receives the unstructured data and the data types information schema221from the retrieval engine210, and then constructs a graph data structure. (SeeFIG. 6a). The graph data structure, which consists of a collection of hosts (i.e., vertices) and connections (i.e., edges), is used to organize and store data for each snapshot30. The data types information schema221specifies where the data received from the retrieval engine210belongs in the graph data structure. This is achieved by defining the data type name (i.e., connection or host) used to create the vertices and edges in the graph data structure. The graph configuration settings231also can be used to specify which data type is used as the base data layer by the graph generator230, as well as define the max buffer size for system memory and the minimum sleep time used by the server1when transmitting the snapshot to the client400.

The overlay generator240adds discrete data to the graph structure generated by the graph generator230. Each element of the discrete data additional information55describes a specific node or edge (each being a graph entity element) in the graph. The overlay generator240overlays the specific graph entity element (i.e., node or edge) with the discrete data additional information55.

A client handler280can be utilized to communicate with the client400over the network300. The client handler280dispatches commands received from the client400to the retrieval engine210, and the client handler280delivers the constructed graph data structure to the client400in response to a snapshot request. An embodiment of a network protocol used by the client handler280for communication between the server200and client400is illustrated inFIG. 7.

The client application400preferably is executed on a computing device capable of data processing operations and generating a 3-D visual display, such as a personal computer or workstation. The client400can comprise a network worker401, a file worker402, a communicator405, a 3D objector generator410, a filter420, a three-dimensional layout module430, a movement module440, and a graphical user interface module450.

The network worker401establishes a connection to the server200, sending commands to and receiving data from the server's client handler280. The network worker401receives the graph data structure from the client handler280, de-serializes the data contained within the snapshot, and then dispatches events to the communicator405. Access to the network worker401preferably is isolated and transparent. The file worker402is adapted to read data from recorded files281(e.g., recorded simulations). Access to the file worker402preferably is isolated and transparent.

The communicator405receives data from the network worker401and file worker402. In addition to managing both incoming data and outgoing data, the communicator405provides high level abstraction (i.e., the communicator hides the source of data from the other client-side modules) to the data received from the network worker401or file worker402. The communicator405also is responsible for spawning actors (i.e., nodes and connections) in the 3-D environment. The communicator405preferably features a data settings interface221configurable by a user of the system1to reduce the incoming data set to particular metadata types, thereby allowing the analyst to focus on a particular type or attribute of network data flow.

The 3-D object generator410and the three-dimensional layout module430collectively function to create a three-dimensional environment. Commercially available software, such as Unreal Engine® (Epic Games, Inc.), can be utilized as the 3-D object generator410and layout module430. In a preferred embodiment, the 3-D object generator410creates a three-dimensional environment where each type of network traffic data received from the graph generator230is assigned a three-dimensional object-such as spheres, cubes, pipes, or other 3-D objects. Attributes of the network traffic data can be distinguished in the 3-D environment by a color, a size, a position, or changing dynamic values of these three-dimensional objects. For example, all hosts on the private network can be represented as purple spherical icons in space, all hosts on the public network can be represented as green spherical icons in space, all hosts on the external network can be represented as yellow spherical icons in space, and the traffic between each computer host can be represented as colored lines or pipes within space. The 3-D object generator410may also assign a certain color, shape, size, position, or a changing dynamic value (e.g., flash, pulse, spin, etc.), which can be adjusted by the analyst through the user interface450) to each type of overlay metadata received from the data overlay generator240. For example, a particular network host having a network security alert (i.e., alert data types metadata) associated with it can be represented within the 3-D environment as a red, pulsing sphere, while the unaffected network hosts will remain being represented as blue spherical icons. This allows a human analyst to visually inspect the environment surrounding the computer host(s) associated with the security alert. The analyst can then quickly backtrack any intrusion and block out the attacker by modifying firewall rules or taking other remedial action.

The layout module430provides a means for displaying the hosts or “nodes” in virtual space according to a specified strategy. For example,FIG. 1ais an illustration of an exemplary “sandwich” or “layered” 3-D mixed reality environment generated by the layout module430, whileFIGS. 1b-1fillustrate an exemplary “onion” 3-D mixed reality environment generated by the layout module430.

The movement module440controls the end user's movement within the 3-D environment. In one embodiment, the movement module440is configured for receiving user input from 3-D peripheral devices such as virtual reality head-mounted displays (e.g., Oculus Rift®—Oculus VR, LLC). In alternative embodiments, the movement module440is configured for receiving user input from a standard keyboard and mouse or a gamepad. Commercially available software, such as Unreal Engine® (Epic Games, Inc.), can be utilized to create the layout module430and the movement module440.

The client400can further comprise a graphical user interface450, which allows the analyst to interact with the data visualized in the 3-D environment through the layout and movement modules430,440, as well as access and customize the appearance settings411and the filter settings421. The appearance settings411allows a user/analyst to configure the 3-D object generator410to specify the desired representation of data types and attributes within the 3-D environment. For example, in an embodiment the appearance settings411can be configured to allow a user/analyst to change: the color assigned to the network topology; the host and the connection color opacity (e.g., when in grey scale); the color assigned to a specific communication protocol (HTTP, FTP, DNS, etc.); the maximum and minimum of host bloom effect intensity (as dictated by the amount of data transmitted), and other visual personalizations.

The filter settings421allows a user/analyst to configure the user interface filter420to modify the visualization created by the 3-D object generator410by refining and decluttering the three-dimensional environment. For example, when an analyst identifies a particular security threat in the environment which calls for further analysis, the analyst could adjust the filter settings421to focus on the particular host(s) associated with the alert over a particular time period (e.g., N+/−5 min). This adjustment of the filter settings421will automatically adjust the appearance settings411of the 3-D object generator410, causing the unelected nodes to change size, shape, or color so as to imperceptible (e.g., invisible or greyscale) to the analyst. Because adjustments to the user interface filter420dynamically adjusts the appearance settings411, the analyst has control over the display of network and overlay data in the 3-D environment.

An exemplary graphical user interface450may provide the analyst with a plurality of screens to allow for the adjustment of the filter settings221and the appearance settings222. Referring toFIG. 2e, a sample embodiment of a time and intervals interface screen15is shown, which allows the analyst to adjust the system's configuration settings relating to the mode of the system (e.g., the real time mode button16or playback mode button17), cycle between the previous and next snapshots (e.g., the previous snapshot button18or the next snapshot button19), or set the snapshot aperture (e.g., the snapshot aperture input box20). Referring toFIG. 2f, a sample embodiment of a configuration interface screen22is shown, which allows an analyst to adjust the appearance settings222(connection colors, topological properties, etc.). Referring toFIG. 2g, a sample embodiment of a ports and protocols interface screen24is shown, which allows an analyst to select and deselect the visualization of certain port types and protocol types in the 3-D environment. Lastly, referring toFIG. 2h, a sample embodiment of a sensors interface screen26is shown, which allows an analyst to select and deselect the visualization of data collected by certain sensors in the 3-D environment. For example, if an analyst desired for the system to only visualize the network traffic overlaid with IDS alerts, the endpoint sensors, log aggregator sensors, and cyber-intelligence sensors could be deselected. In a preferred embodiment of the system1, the user interface450is selectively superimposed over the populated 3-D environment to allow an analyst to quickly access the filter settings221and the appearance settings222.

FIG. 3is a flow diagram depicting an exemplary process carried out by the system1to provide three-dimensional visualization of network traffic overlaid with discrete data pertaining to one or more computer hosts or connections in the network, thereby allowing a human analyst to quickly inspect and understand the monitored computer network. The system1is particularly useful for the analysis and investigation of security alerts, as the system1can display exponentially more usable data to human analysts than prior art systems.

Referring toFIG. 3, the system1receives a request in step500. In a preferred embodiment utilizing a server-client solution, a user/analyst initiates the request through a client400, which is then communicated to the server200over the network300. Upon receiving the request, the system1will retrieve collected network traffic metadata and discrete data (e.g., security alerts) from the system database102(step550). In step600, the system1will process the data by building a graph data structure using the network traffic metadata, and then embedding within the graph data structure one or more layers of network traffic additional information54and, if available, one or more layers of discrete data additional information55.

Next, the system1will generate and populate a three-dimensional environment to allow the user to visualize the network traffic and overlaid additional information (step700). The system1will spawn nodes (i.e., network hosts) and connections between nodes in the 3-D environment, assigning a three-dimensional object to each. The system1will also assign attributes (e.g., a certain color, shape, size, position, or a changing dynamic value) to each type of discrete data overlay information. Optionally, in step800, the three-dimensional environment can be further refined and decluttered by applying user interface filter(s) set by the user/analyst. For example, the system1can be adjusted to focus on the particular host(s) associated with an alert over a particular time period. Lastly, the system1will display the network traffic and overlay information in the three-dimensional environment (step900).

FIG. 4shows an exemplary process carried out by the server of the present invention upon receiving a request from a user. In step501, the application server200receives a request from the user via the client400. The server200first validates whether the client is approved to solicit data from the server; e.g., whether the client400has the proper software, API, login/password codes, etc. (step502). If the client request is validated, the server200will process the request by analyzing the command value in the packet header (step503). If the client request is invalid, the server200will send an invalid request notification (step504) to the client400.

The system1can be configured to handle a variety of request types, including a configuration request510, a keep alive request512, a goodbye request514, a real time mode request516, a replay or playback request518, a snapshot aperture change request520, a replay speed change request522, a next snapshot request524, a previous snapshot request526, a pause request528, and a resume request530. In a preferred embodiment, each request inputted by the user/analyst is transmitted by the client's communicator405to the server's client handler280, which in turn instructs the retrieval engine210to retrieve the collected and process the metadata. The client handler280will then transmit the data to the client400for generation and population of the three-dimensional environment.

If the request is a configuration request (step510), the server200will access the global client configurations, the client-specific configurations, and the server protocol version being utilized by the server200. The server200will transmit the configuration settings to the client, and then await a new client request from the client.

If the request is a request to keep the sever-client connection open (step512), the server will keep the connection alive, send a confirmation to the client400, and then await a new client request from the client. If the request is a request to close the sever-client connection (step514), the server will close the connection and then await a new client request from the client.

If the request is a request to stream data via real time mode (step516), the server200will retrieve and process the network metadata and discrete data (which is being continuously collected by the data collector(s)101) on a real-time basis, thereby allowing for the real-time display of the network metadata and discrete data in the three-dimensional environment by the client400.

If the request is to stream a data replay (step518), the server200will retrieve and process the network metadata and discrete data for a particular time interval specified by the user/analyst. The client400will then stop displaying the previous replay or previous snapshot aperture, and start displaying the network metadata and discrete data for the specified time interval in the three-dimensional environment.

The snapshot aperture feature of the system1allows the user/analyst to set a time window (i.e., the snapshot aperture) which defines the start time and end time of computer network activity to be visualized by the system1. If the request is to update the snapshot aperture, the server200will update the start time and end time for the current replay (step520). Next, the server200will retrieve and process the network metadata and discrete data within the new snapshot aperture. The client400will then stop displaying the previous replay, and start displaying the updated replay with the newly updated snapshot aperture.

The system1can be configured to handle a variety of requests relating to the display of the network metadata and discrete data in the three-dimensional environment. If the request is to adjust the replay speed (step522), the server200will send snapshots to the client at the newly defined frequency (the speed of the replay), and then await a new client request from the client. If the request is a Next Snapshot request (step524), the server200will skip to the next snapshot, and then await a new client request from the client. If the request is a Last Snapshot request (step526), the server200will replay the previous snapshot, and then await a new client request from the client. If the request is a request to pause (step528), the server200will close the current streaming of the network metadata and discrete data in the three-dimensional environment, and then await a new client request from the client. Lastly, if the request is to resume (step530), the server200will retrieve and process the network metadata and discrete data for the next snapshot, based upon the existing settings. The client400will then stop displaying the previous replay or previous snapshot, and start displaying the network metadata and discrete data of the requested snapshot in the three-dimensional environment.

FIG. 5shows an exemplary process carried out by the server of the present invention for retrieving network metadata and discrete data and processing the same. Upon receiving a request from the client400for a snapshot, the server200will first inquire whether the system1has been set to real time mode (seeFIG. 4, step516) or replay mode (seeFIG. 4, step518).

If the system1is set to replay mode, the server200in step554will set the snapshot aperture—the time interval defined by the start date time35and the snapshot end date time36as specified by the user/analyst. In step555, the server's retrieval engine210will then communicate with the network database102and request network traffic metadata and discrete data associated with the specified search time interval. If network traffic metadata is determined to exist within the specified search time interval (step599), the graph generator230will process the retrieved network traffic metadata by constructing a graph data structure and overlaying the network traffic additional information54as shown in greater detail inFIG. 6a(step600). Next, if discrete data is determined to exist within the specified search time interval (step649), the overlay generator240will process the retrieved discrete data by overlaying the discrete data additional information55over the graph vertices and edges as shown in greater detail inFIG. 6b(step650). Lastly, the server200will remove the oldest snapshot from the replay buffer, and then insert the new snapshot in the replay buffer for transmission to the client400(step691).

If the system1of the present invention is set to real-time mode, the server200will retrieve and process the network and overlay metadata from the network database102on a real-time basis, thereby allowing for the real-time display of the network and overlay data in the three-dimensional environment by the client workstation400. In step553, the server200will set the snapshot aperture for real-time display of the network environment by starting the real-time mode timer and retrieving the sleep time parameter from the graph configuration file231. Using this data, the server200is capable of setting the snapshot aperture for real-time capture of the network traffic metadata and discrete data. Next, the server200queries the system database102for all documents within the real-time snapshot aperture, and then the server200retrieves and processes the network traffic metadata and discrete data as described above with respect to steps555through691.

In a preferred embodiment, the system1may also employ a process for ensuring that the real-time timer remains properly synchronized in order to capture subsequent real-time snapshots. In order to keep the real-time timer properly synchronized, the server200will evaluate how much time passed while the previous snapshot was being generated, and then the server200will determine if more time passed than the sleep time value provided by the graph configuration file231. If the sleep timer was exceeded, the server200changes the start time on the next snapshot in order to prevent a gap in the retrieval of data. If the timer has not exceeded the sleep time, then the server200will wait until the timer reaches the sleep time value before starting to retrieve and process the next snapshot for real-time display by the client400.

FIG. 6ashows an exemplary process carried out by the server of the present invention for constructing a graph data structure using the collected network traffic metadata and then embedding one or more layers of network traffic additional information54into the graph data structure. While an exemplary process for constructing the graph data structure is shown inFIG. 6aand described herein, one skilled in the art will readily recognize that alternative processes can be utilized to represent the collected metadata in a graph structure.

The retrieval engine210retrieves unstructured data from the system database102upon receiving a request from a client400, loads the data types information schema221files into system memory, and parses the data using standard JSON encoding/decoding functions by evaluating the data using the data types information schema221. The graph generator230receives this parsed data and the data types information schema221from the retrieval engine210and produces an intermediary data array. The graph generator230then processes the data using the data types information schema221for a given document type to create a graph data structure used to organize the network traffic metadata. The graph data structure is populated by vertices (which represent computer hosts) and edges (which represent connections) to visualize the network environment. The particular metadata types used to create the vertices and edges in the graph structure are shown inFIG. 2b. The additional information overlay generator240cooperates with the graph generator230to include with one or more layers of network traffic additional information54about the computer hosts and connections represented in the graph data structure. And as further described with respect toFIG. 6b, discrete data additional information55can also be embedded into the graph data structure, thereby allowing an analyst to visualize alerts and other information superimposed over the 3-D network environment.

Referring now toFIG. 6a, the server's graph generator230will first inquire if the dataset received from the retrieval engine is empty (step601). If data is available, the graph generator230will extract the normalized fields40data according to the data type information schema221and insert the data into an intermediary data array (step602). Next, for each non-normalized data field read by the graph generator230(e.g., network metadata details such as number of packets sent, bytes transmitted, etc.), the graph generator230will assign the appropriate group ID51and group name52to the data according to the data type information schema221, and then it will insert the data into the overlay information map50-anested array (step604). The intermediary array and nested sub-arrays are built in an iterative fashion for each document that is processed (steps602-604), with the aggregated data available in the intermediary array (which includes the nested sub-arrays) being used to generate the graph data structure.

The intermediate data array generated in steps602through604can include edge data, vertex data, or edge and vertex data. In a preferred embodiment, the data type information schema221can be configured to identify the intermediate data array with a layer type value, namely: Type 1 for arrays consisting of edge data only; Type 2 for arrays consisting of vertex data only; and Type 3 for arrays consisting of both edge and vertex data.

In step605, the graph generator230will evaluate the intermediary array generated in steps602through604and evaluates the data layer type. If the intermediary array is identified as a Type 1 or Type 3 data layer type, the graph generator230will search for an existing edge on the graph data structure using the normalized field data40organized within the intermediary array (step607). If an existing edge is found, the graph generator230in step611will extract from the intermediary array the network traffic additional information54that is associated with the connection, and then the graph generator230will add the network traffic additional information54to the graph edge. If an existing edge is not found, the graph generator230will first add a new edge to the graph (step608) and then add the network traffic additional information54to the graph edge (step611).

Next, the graph generator230will determine if any new vertices need to be added to the graph structure by using the normalized field data40to search for an existing vertex associated with the edge in the graph (step613). If an existing vertices is found, the graph structure will be updated (step619). If no existing vertices are found, in step615new vertices will be added to the graph data structure (representing both computer hosts involved in the connection), associated metadata from the normalized fields40will be added to the vertices within the graph data structure (step617), and then the graph structure will be updated (step619).

Once the graph structure is updated, the graph generator230will search for existing vertices within the graph data structure based upon the normalized fields data40(step621). If no existing vertices are found, the graph generator230will initialize array values to remove the present document from the document list (step625), and then will again inquire whether the document list is empty (step601). However, if existing vertices are found, the graph generator230will extract the network traffic additional information54that is associated with each computer host, add the network traffic additional information54to the graph vertices (step623), remove the present document from the document list (step625), and then will again inquire whether the document list is empty (step601).

If the data array is identified as a Type 2 data layer type, the graph generator230will bypass steps607through619since a data array having vertex-only information does not provide a basis for creating a new edge (i.e., a connection). When processing a Type 2 data layer type, the graph generator230will cycle through steps621-625to determine if network traffic additional information54to the graph vertices should be added to one or more existing vertices. Once completed, the graph generator230will remove the present document from the document list (step625), and then will again inquire whether the document list is empty (step601).

FIG. 6bshows an exemplary process carried out by the server of the present invention for overlaying the graph data structure with discrete data additional information55derived from the discrete data. Referring toFIG. 6b, the server's overlay generator240will first inquire if the dataset received from the retrieval engine is empty (step651). If data is available, the overlay generator240will extract the normalized fields40data according to the data type information schema221and insert the data into an intermediary array (i.e., the overlay information map50)(step652). The normalized fields in the intermediary array are utilized by the overlay generator240to associate the discrete data additional information55with the appropriate edge or vertex in the graph data structure. Next, for each non-normalized data field read by the overlay generator240, the overlay generator240will assign the appropriate group ID51and group name52to the data according to the data type information schema221, and then it will insert the data into the overlay information map50as a nested array (step654). The overlay information map50's intermediary array and nested sub-arrays are built in an iterative fashion for each document that is processed (steps652-654), with the aggregated data available in the intermediary array (which includes the nested sub-arrays) being used by the overlay generator240to populate the graph data structure with discrete data relevant to the computer hosts and connections active within the monitored network.

Like the graph generator230, the intermediary data arrays generated by the overlay generator240in steps652through654are identified with a layer type value, specified in the data type information schema221. In a preferred embodiment, the schema221defines three possible types of data layer arrays: Type 1 (edge data only); Type 2 (vertex data only); and Type 3 (both edge and vertex data).

In step655, the overlay generator240evaluates the intermediary data array generated in steps652through654and evaluates the data layer type. If the data array is identified as a Type 1 or Type 3 data layer type, the overlay generator240will search for an existing edge on the graph based upon the normalized fields (step657). If an existing edge is not found, the overlay generator240will remove the present document from the document list (step665), and then will again inquire whether the document list is empty (step651). However, if an existing edge is found, the overlay generator240in step659will extract the discrete data additional information55that is associated with the connection from the intermediary array, and then the overlay generator240will add the discrete data additional information55to the graph edge (step659).

Once the discrete data additional information55is added to the graph edge in step659, the overlay generator240in step661will next search for existing vertices on the graph based upon the normalized fields (step688). If an existing vertex is not found, the overlay generator240will remove the present document from the document list (step665), and then will again inquire whether the document list is empty (step651). However, if an existing vertex is found, the overlay generator240in step663will extract the overlay additional information from overlay information map50that is associated with the computer host(s), and then the overlay generator240will add the discrete data additional information55to the each vertex. The overlay generator240will then remove the present document from the document list (step665), and then will again inquire whether the document list is empty (step651).

If the data array is identified as a Type 2 data layer type, the overlay generator230will bypass steps657through659since a data array having vertex-only information does not provide a basis for adding any overlay data to an existing edge (i.e., a connection). When processing a Type 2 data layer type, the overlay generator240will cycle through steps661-665to determine if any discrete data additional information55should be added to an existing vertex. Once completed, the overlay generator240will remove the present document from the document list (step665), and then will again inquire whether the document list is empty (step651).

FIG. 7shows an exemplary network communication protocol that can be utilized by the system1when a server-client solution is employed. The network communication protocol is preferably configured to allow large amounts of data to be transmitted between the server200and the client workstation400. While an exemplary network communication protocol is shown inFIG. 7and described herein, one skilled in the art will readily recognize that alternative communication protocols can be utilized to transmit data between the server200and client workstation400.

Referring now toFIG. 7, the network communication protocol700can be utilized to transmit and receive data from a data file or from a network socket. When the data is being transmitted from the server200to the client400via the network socket (e.g., the transmittal of the graph structure by the client handler280to the network worker401), the network worker401first will inquire whether the client connection is enabled (step704).

If enabled, the network worker401in steps705-708will initiate a subroutine to determine if the client-server connection is still alive. The network worker401features a timer which counts time (in seconds) since its last operation (step705). In step706, the network worker401will inquire whether the elapsed time since the last operation is greater than N seconds to verify that the connection is alive (step705). In a preferred embodiment, N is set to three seconds, while the server200is configured to terminate the client-server connection after five seconds if the server has no outstanding operations for the particular client400. If the elapsed time since the client400's last communication with the server200exceeds N seconds (e.g., three seconds), the client400in step707will send a “keep alive” command to keep the connection towards the server200alive, reset the time counter (step705), and then will again inquire whether the elapsed time since the last operation is greater than N seconds (step706).

If the elapsed time since the client400's last communication with the server200is less than N seconds (e.g., three seconds), the network worker401will inquire whether there are any commands that need to be sent to the server200(step708). If there are commands that need to be sent, the network worker401will serialize the command and transmit it to the server200(step710), and then the network worker401will again query whether a connection is enabled (step704). If there are no commands that need to be sent by the client400to the server200, the network worker401will wait for data available in the socket (step712).

If a data packet is available in either a data file or in the network socket, the network worker401will retrieve the data packet from the server200and deserialize the data packet to the memory buffer (step714). Next, the network worker401will inquire whether data is contained within the memory buffer (step715). If no data is located in the memory buffer, the network worker401will discard the data packet (step716) and momentarily sleep (step730). However, if data is located, the network worker401will evaluate the data header command value and execute the command (step718).

In step720, if the command is to close the connection, the network worker401will close the client-server connection (step721), reset the internal communication variables by deleting the temporary socket array received in step714(step728), and then momentarily sleep (step730).

In step722, if the command is to receive a serialized data payload (e.g., a graph data structure representing the network traffic metadata and discrete data for the time interval), the network worker401will deserialize the payload by converting the stream of bytes contained in the packet payload into the corresponding object definition (e.g., FNetNetworkStatus) (step723); convert the snapshot data received from the server into a format readable by the 3-D object generator410(step724); will pass the snapshot to the 3-D object generator410via the communicator405(step725); reset the communication protocol variables (step728); and then momentarily sleep (step730). In step722, if the command is not to receive a serialized data payload, the network worker401will process the “keep alive” signal (step726), reset the communication protocol variables (step728), and then momentarily sleep (step730).

FIG. 8shows an exemplary process by which the client initializes the runtime components that receive, process, and visualize data via the network worker401, the file worker402, the communicator405, the 3-D object generator410, three-dimensional mixed reality layout module430, and the user interface450.

Referring now toFIG. 8, in step740the client400will initialize event dispatchers responsible for communication between the data handler modules, and the visualization modules that present the environment to the user. The event dispatchers are necessary for communication between the data handling modules (i.e., the network worker401and the file worker402) and the user interface450. Next, the client400will initialize the finite state machine in order to keep track of the internal state of the communicator405(step750).FIG. 12depicts an exemplary embodiment of a finite state machine suitable for use with the system1. The client400will then initialize the queues, which will be filled and emptied with actors (vertices/hosts) that are preferably spawned asynchronously in order to avoid exceeding the system1client's processing capabilities (step760). In step770, the client400will set the state of the finite state machine to NORMAL, and then the connection renderers (i.e., the modules responsible for connection rendering) will be initialized (step780). As with actor queues, instanced rendering is used to decrease the geometry count and to reduce the computational load on hardware (e.g. video graphics cards). Once the initialization process has been completed for client processes, the 3-D visualization of the network environment can be generated and populated using the graph data structure received from the server200.

FIGS. 9-11show an exemplary process that can be utilized by the system1for generating and populating the three-dimensional environment.FIG. 9depicts an embodiment of a process for parsing a new snapshot and preparing the 3-D environment for the spawning of new nodes and connections in the simulation.FIG. 10depicts an embodiment of a process for spawning nodes (i.e., graphical representations of computer hosts) in the simulation.FIG. 11depicts an embodiment of a process for spawning connections (i.e., graphical representations of network traffic between two computer hosts) in the simulation.

Referring toFIG. 9, the communicator405first implements a temporary lock to prevent the communicator405from spawning new nodes and connections while the snapshot is being parsed (step802). Next, the communicator405will retrieve the next snapshot (i.e., the graph data structure representing the network traffic metadata and discrete data for a specified time interval) (step804), parse the snapshot metadata by populating the user interface fields (e.g., start date time35, end data time36, sensor name list33) (step806), and then dispatch the metadata information (e.g., start, end, number of hosts, live or replay) to the user interface450for display to the analyst (step808). The communicator405then will empty the sensor map (i.e., a list of sensors displayed in the user interface450) from the previous snapshot to ensure the newly generated environment reflects information from the present snapshot only (step810), enqueue nodes from the snapshot by pushing incoming snapshot data-which has been parsed by the network worker401into a format readable by the 3D object generator410—to the corresponding spawning queue (step812), and remove unused nodes from the simulation that had been previously spawned by the 3D object generator410for the last snapshot (step814). Next, the communicator405will enable the spawning of connections in the simulation (step816) and remove the temporary lock so that nodes can be spawned (step818). In step820, the communicator405will query whether there are any additional snapshots in the queue, with steps804through820being repeated by the communicator405until there are no additional snapshots in the queue.

Referring now toFIG. 10, a process for spawning nodes in the simulation is shown. In step822, the communicator405will verify that the client400is unlocked to allow the communicator405to spawn new nodes in the simulation. If unlocked, the communicator405will inquire whether there are any nodes to spawn in the spawning queue (step824). If the spawning queue is empty, the communicator405will proceed to verify that the communicator405is unlocked to spawn new connections in the simulation (step848), will remove unused sensors from the sensor map (step850), add any new sensors to the map (852), and then issue a command to the finite state machine to switch to the New Snapshot state (step856).

If the spawning queue is not empty, the communicator405will initiate a process of spawning new nodes in the simulation. The communicator405first will remove a node from the queue (step826) and then spawn the node in the 3-D environment (step828). Next, the communicator405will add the node to a present node map—an intermediary data structure populated in an iterative fashion by the communicator405until the spawning queue is emptied (step830). In step832, the communicator405will parse the IP address and network location (i.e., whether the node is within the private network, public network, or external network) for the node. If the node has a publicly routable IP address, the communicator405will access the overlay information map50to obtain the latitude and longitude coordinates for the node (step836). The node will then be assigned to the appropriate visualization layer based upon whether the node is located within the private, public, or external network (or any other visualization layer defined by the system1), and the node will be assigned to the appropriate position defined by the latitude and longitude coordinates for the node contained within the network traffic additional information54pursuant to step836, or at a default position based upon the node's visualization layer if the latitude and longitude coordinates are not available in the network traffic additional information54. In step840, the communicator405will add the node to the user interface sensor map (i.e., the listing of sensors visible, through the user interface450), and then in step842the communicator405will inquire whether there are additional nodes present in the spawning queue (step842). If additional nodes are present, steps826through840will be repeated in an iterative fashion until the spawning queue is emptied.

Once the spawning queue is emptied, the communicator405will assign each node located within the present node map to the appropriate region (step844), and then the 3-D object generator410will regenerate the visualization layers in order to position the nodes within the 3-D simulation (step846). Next, the communicator405will proceed to verify that the communicator405is unlocked to spawn new connections in the simulation (step848), will remove unused sensors from the sensor map (step850), add any new sensors to the map (852), and then issue a command to the finite state machine to switch to the New Snapshot state (step856).

Referring now toFIG. 11, an exemplary process is depicted for building connections within the 3D simulation and overlaying the nodes and connections with the network traffic additional information54and the discrete data additional information55contained within the snapshot. In step858, the communicator405will disable communication between the user interface450and the communicator405while new connections are spawned pursuant to the process described herein. Next, the communicator405will clear the alert signatures map (i.e., the listing of alerts visible through the user interface450), thereby removing from user interface450all alert information from the previous snapshot (step860).

In step862, the communicator405will get the next connection from the snapshot graph structure. In step864, the communicator405will retrieve the nodes associated with this connection from the present node map. If the source and destination host metadata is determined to be valid based upon the normalized field data (step866), the communicator405will spawn the connection (e.g., a three-dimensional pipe representing traffic between two nodes) and assign the connection a color based on the destination port number associated with the connection (step868). In step870, the communicator405will overlay the connection with network traffic additional information54and, if available, with one or more layers of discrete data additional information55. Next, the communicator405will inquire whether the connection array is empty (step872), with steps862through870being repeated until the connection array is emptied. Once emptied, the communicator405will overlay the nodes with the network traffic additional information54and discrete data additional information55(step873), fill the user interface map with alert signature information (step874), fill the user interface map with sensor information (step876), and then re-enable the user interface450to communicate with the communicator module405(step878).

FIG. 12shows the various states of the generated 3-D environment and the transitions used by the system into and out of these states. The communicator405's finite state machine cooperates with the 3-D generator410to handle the process of generating the 3-D environment, with the display of the 3-D environment being dynamically adjusted as new snapshots are received by the client400and as filter settings are inputted by an analyst.

Referring toFIG. 12, the Normal state902is the starting state of the finite state machine whereby all nodes and connections in the present snapshot are visualized, and the Focused state908is an auxiliary state assumed by the system1when the user has selected a node on which to focus. Whether in the Normal state902or in the Focused state908, the system1is configured to allow for the seamless transition to new snapshots and the regeneration of connections within the three-dimensional environment.

When in the Normal state902and a new network snapshot is available, the New_Snapshot transition921will communicate the arrival of a new network snapshot, with Normal New Snapshot904being an intermediary state of the finite state machine associated with the arrival of the new network snapshot. Regen_Conn transition922is a transition of the finite state machine used to communicate the need of regenerating connections, with New_Network906being an intermediary state of the finite state machine associated with the regeneration of all connections between hosts displayed in the three-dimensional environment. Regen_Done transition923is a transition of the finite state machine used to communicate that the regeneration of the connections is complete, with the system1then re-assuming the Normal state902.

The Focus_On transition924is a transition of the finite state machine used to communicate the selection of a host and the transition to the Focused state908, while the Focus_Off transition925is a transition of the finite state machine used to communicate the deselection of a host and the transition back to the Normal state902. When in the Focused state908and a new network snapshot is available, the New_Snapshot transition921will communicate the arrival of a new network snapshot, with the Focus New Snapshot state912being an intermediary state associated with the arrival of a new network snapshot while maintaining the focused status. Regen_Conn transition922is a transition of the finite state machine used to communicate the need of regenerating connections, with the Focus New Network906state being an intermediary state of the finite state machine associated with the regeneration of all connections between hosts displayed in the three-dimensional environment while maintaining the focused status. Regen_Done transition923is a transition of the finite state machine used to communicate that the regeneration of the connections is complete, with the system1then re-assuming the Focused state908.

Referring now toFIGS. 13 and 14, the system1is particularly adept at providing contextual and intuitive visualization of security data in order to prioritize security response activity. Data collector's such as intrusion detection systems (IDS's) can be utilized to obtain data pertaining to a security alert. The system1visualizes the network environment, thereby providing the proper context of security events so that the analyst can investigate the security alert and develop an appropriate response for addressing any intrusion(s).

One investigative tool enabled by the system1which is particularly useful for investigating security alerts is referred to herein as the path feature. An embodiment of the path feature is shown inFIGS. 13a-13e. The path feature allows a user to select specific objects (e.g. hosts, connections, user accounts, login events, etc.) within the 3-D environment and add these objects to the path listing. Once added, the objects are highlighted in the environment and represent the path taken by an attacker during the potential cybersecurity data breach (or intrusion). The path list is a listing of computers, user accounts, data, and other assets within the computer network environment that the attacker leveraged in order to intrude into the network. This path list and associated information can be used by cybersecurity analysts to identify all computer network assets that require remediation in order to stop an intrusion and remove an attacker's presence from the network environment. An analyst can initiate the path feature by adjusting the filter settings421of the client400, which will engage the communicator405and the 3D object generator410to create a listing of selected hosts potentially affected by the intrusion.

When using the path investigative tool, the analyst has the option to add selected nodes to the path (FIG. 13c), and remove selected nodes from the path (FIG. 13d).FIG. 13ais an illustration of an embodiment of the generated 3-D environment whereby a node is shown having a security alert associated with it and an analyst has added two nodes to the path.FIG. 13bis an illustration of an embodiment of a graphical user interface showing a listing of nodes in the current path list. In a preferred embodiment, the unselected nodes and their associated connections will be shown in greyscale in the environment in order to declutter the user interface450.

Referring toFIG. 13c, an exemplary process is shown for adding a node to the path of a potential network intrusion being investigated by an analyst. In step941, the communicator405will acknowledge the host selected by the analyst using the user interface450in the current snapshot. Next, in step943, the communicator405will inquire whether the path list is empty (step943). If the path list is empty, the communicator405will add the selected host to the path (step948), load the path mode for the selected host and set the IP address for display (step949), and then notify the user interface (step960).

If the path list is not empty, which means that a first host has already been selected and added to the path by the analyst, the communicator405will inquire whether the second selected host has a community-of-interest value indicating a direct connection with the first host (step944). If false, the communicator405will clear the path (step945), will add the selected host to the path (step948), apply a texture (i.e., an image, such as a concentric circle) to the selected host (step949), and then notify the user interface450(step960). If true, the communicator405will then inquire whether the path already contains the selected node (step (947). If true, the communicator405will cause the selected node to be removed from the path (step947) and will then update the user interface (step960). If false, the communicator405will add the selected host to the path (step948), load the path mode for the selected host and set the IP address for display (step949), and then notify the user interface (step960).

Referring now toFIG. 13d, an exemplary process is shown for removing a node/host from the path of a potential network intrusion being investigated by an analyst. In step951, the communicator405will find the selected node in the path index. Next, the communicator405will get the next host in the selected path (step952), load the path mode for the selected host and set it to hide the IP address from display (step953), and then inquire whether the selected path is now empty (step954). If the selected path is not empty, the communicator405will repeat steps952through954. If the selected path is empty, the communicator405will remove the node from the path (step955) and update the path list being displayed to the analyst through the user interface450(step960).

FIG. 13edepicts an exemplary quality control process utilized by the system of the present invention for verifying that a connection to be visualized between two hosts exists in a new snapshot. Referring toFIG. 13e, an exemplary process for checking the path of a potential network intrusion is shown. In step931, the communicator405will inquire whether the background workers (i.e., the network worker401and the file worker402) are running an asynchronous operation. If the background workers are not running an asynchronous operation (i.e., the system is not presently spawning nodes and connections in the 3-D environment), the communicator405will get the next host in the path (step932) and then inquire whether the host is present in the snapshot (step933). If the host is in the snapshot, the communicator will repeat steps932through933. If the host is not in the snapshot, the communicator405will remove the host/node from the path (step934), remove the remaining nodes from the path (step935), and then inquire whether the path is empty (step936). If the path is empty, the communicator will end the operation. If the path is not empty, the communicator405will repeat steps932through936.

Another investigative tool enabled by the system1which is useful for investigating security alerts is referred to herein as the community-of-interest feature. An embodiment of the community-of-interest feature is shown inFIGS. 14a-14d. The community-of-interest (COI) feature allows an analyst to use link analysis to examine all computers that have directly communicated with a selected computer host, as well as all computers indirectly linked to the selected computer host by orders or degrees of separation. For example, an analyst can select a computer host in the 3-D environment, such as a computer host exhibiting an intrusion alert, and use the COI feature to visualize all hosts that communicated with the selected computer host within the snapshot timeframe. By default, the COI value is set to 1, which means that only computers with direct connections to the selected computer will be displayed within the 3-D environment. If the analyst increments the COI value to 2, all computers with one degree of separation from the subject computer (e.g. computers that communicated with computers that communicated directly with the selected computer) will be displayed as well. Further increments will highlight computers with higher degrees of separation from the selected computer host. In this manner, an analyst can use link analysis to display computers that have communicated within a computer network to a certain degree of separation, which is useful to identify a population of computer hosts that may be indirectly implicated in a cybersecurity intrusion.

Two values specific to the COI feature are used to flag data evaluated by the COI function: “crossed” and “visited”. “Crossed” flags indicate that a connection between two hosts in the current snapshot has been evaluated, and “visited” flags indicate that a host in the current snapshot has been evaluated while processing COI selection requests. These flags allow the system1to determine which hosts will be highlighted once a COI selection is made by the user in the process detailed below.

FIG. 14dshow an exemplary process enabled by the system1of the present invention for applying user interface settings in a manner allowing an analyst to use the COT investigative tool. When an analyst is utilizing the COI feature, the client400will be in the Focused state908—an auxiliary state allowing the user to focus on a selected a node. Referring toFIG. 14d, the communicator405first will mark all hosts not visited by the analyst (step971) and then mark all connections not crossed (step972). Next, the communicator405create a queue for a Breadth First Search (i.e., is an algorithm for traversing or searching tree or graph data structures) process (step973) and mark a selected node as the starting point (step974). In step975, the communicator405determines if it has processed all nodes in the current breadth level. If so, it will increase the current breadth level (step976) and then check if the current breadth level is the same level as the user selected COI level (step977). If it has not reached the target COI level, in step978the communicator405will get the next node in the queue, and then get its adjacent nodes (step979). Next, the communicator405will inquire whether the adjacent node has already been visited (step980). If the adjacent node has not been visited, the node will be added to the queue (step981) and then the current connection will be marked as crossed (step982). If the node has already been visited, the connection will be marked as crossed without adding the node to the queue (step982). Next, in step983, the communicator405determines if the adjacent node list is empty, and if not, repeats steps979-982until the adjacent queue is completely processed before marking the current node as visited in step984.

In step985, the communicator405will inquire whether the queue is empty. If not, steps975through984will be repeated until the queue is emptied. Once the queue is empty, or if the current COI level is greater than the input COI level (evaluated in step977), the communicator405will inquire whether the number of hosts in the selected path is greater than one (1) (step986). If not, the communicator405will clear the existing connections (step987) and then regenerate all connections between hosts displayed in the three-dimensional environment while maintaining the focused status (step988), and then the COI process ends.

If the number of hosts in the selected path is greater than one (1), the communicator405will get the next host in the path (step990) and then get the connections for the next host in the path (step991). Next, the communicator405will inquire whether the destination node or source node (i.e., the destination or source of the traced connection) is the next host in the path (step992). If yes, the communicator405will highlight the connection (step993), mark the connection as crossed (step994), and then inquire whether the list of connections for the host is now empty (step995). If the destination/source node is not the next host in the path, then the communicator will not highlight the connection and will instead transition immediately to the inquiry whether the list of connections is now empty (step995). If the list of connections is not empty, steps991through995will be repeated. Once the list of connections is emptied, the communicator405will then inquire whether there are any additional hosts in the path (step996), and it will repeat steps990through996until the selected path is emptied.

Many modifications and other embodiments of the invention set forth herein will come to mind to one skilled in the art having the benefit of the teaching presented in the foregoing description and associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.