Abstract:
A method and system for processing network data and displaying the results using a customizable user interface are disclosed.

Description:
[0001]    An improved method and apparatus for processing network data and displaying the results using a customizable user interface are disclosed. 
       BACKGROUND OF THE INVENTION 
       [0002]    The amount of network traffic on the Internet continues to escalate. At the same time, the number of potential security and data integrity threats, such as malware, also continue to escalate. It is becoming increasingly difficult to track the prevalence and spread of these threats due to the overwhelming amount of network traffic. 
         [0003]    What is needed is an improved method and apparatus for collecting and analyzing network traffic to identify potential threats. What is further needed is the ability to filter and/or aggregate the collected data to identify patterns that indicate the presence of potential threats. 
         [0004]    Equally important to data collection and analysis is having a “frontend” user interface that allows a user to quickly understand the data and identify patterns in the data. It is important to allow a user select the type of data of interest and to present the relevant data to the user in a way that a human being can quickly understand. In the prior art, user interfaces would allow a user to filter data by selected criteria. However, prior art user interfaces were limited because the criteria were not updated in real-time as the underlying dataset itself evolved. 
         [0005]    What is needed is an improved user interface that permits a user to customize the user interface to be able to select the data of interest. What is further needed is a user interface that is modified in real-time as the underlying data set changes. 
       SUMMARY OF THE INVENTION 
       [0006]    The aforementioned problems and needs are addressed by a computing device for collecting, filtering, and aggregating network data and displaying the results utilizing a customizable user interface. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0007]      FIG. 1  is an exemplary block diagram of a computing device that collects data from sensors over a network. 
           [0008]      FIG. 2  is an exemplary block diagram of an embodiment of a sensor that collects data over a network. 
           [0009]      FIG. 3  is an exemplary block diagram of other devices used in conjunction with the computing device to analyze data collected over a network. 
           [0010]      FIG. 4  is an exemplary block diagram of a filtering engine and aggregation engine within a computing device. 
           [0011]      FIG. 5  is a block diagram of a prior art system for generating a web page. 
           [0012]      FIG. 6  is an exemplary screen shot of a user interface for a computing device. 
           [0013]      FIG. 7  is another exemplary screen shot of a user interface for a computing device. 
           [0014]      FIG. 8  is another exemplary screen shot of a user interface for a computing device 
           [0015]      FIG. 9  is another exemplary screen shot of a user interface for a computing device. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Data Collection, Aggregation, and Filtering 
       [0016]    An embodiment is now described with reference to  FIG. 1 . Exemplary sensors  21 ,  22 , and  23  are coupled to the Internet  10 . Sensors  21 ,  22 , and  23  each are well-known network traffic collection devices that “sit” on the Internet or a local network and gather data. 
         [0017]    Further description will now be provided for exemplary sensor  21 . It will be understood by one of ordinary skill in the art that the same description applies to sensor  22 , sensor  23 , and other sensors not shown. With reference now to  FIG. 2 , in this embodiment, sensor  21  monitors Domain Name Service (DNS) lookup activity, as would occur when client computer  25  attempts to access a website over the Internet  10 . Typically, the user of client computer  25  will type a URL (or web site address) into a web browser on client computer  25 . Client computer  25  then will initiate a DNS lookup by contacting DNS server  26 , either directly or through an Internet Service Provider or other server (not shown). DNS server  26  will provide client computer  25  with the IP address for the server that hosts the requested URL, in this example, web server  27 . 
         [0018]    Sensor  21  identifies the DNS lookup event initiated by client computer  25 , and it creates a record for the DNS lookup event and optionally can store all or portions of the communication that follows between client computer  25  and web server  27 . The record can include the DNS information, the IP address of client computer  25 , the IP address of web server  27 , a timestamp for all communications, and optionally, the messages (such as HTTP messages) that are transmitted between client computer  25  and web server  27  during the web session. 
         [0019]    With reference again to  FIG. 1 , periodically, sensor  21  will send all or some of the collected data to computing device  50 . Computing device  50  is coupled to the Internet  10  through firewall  30 . Firewall  30  comprises any known hardware and/or software firewall device. Firewall  30  protects the security and integrity of computing device  50 . Sensor  21  can organize and send the data using known data formats, such as an XML file. The data optionally first enters queue  40 , where the data can be further organized and/or filtered before sending it to computing device  50 . 
         [0020]    With reference now to  FIG. 3 , computing device  50  is coupled to storage device  60  and storage device  70 . Optionally, storage device  60  can be configured as long-term storage and can comprise hard disk drives, optical drives, tape drives, flash devices, or any other storage medium. Optionally, storage device  70  can be configured as short-term storage and can comprise flash devices, RAM, or other quickly-accessible storage. Computing device  50  optionally can run a database application, such as MySQL, and can store the data in storage device  70  and/or storage device  60 . 
         [0021]    With reference now to  FIG. 4 , additional detail is provided regarding computing device  50 . Computing device  50  optionally comprises filtering engine  55  and aggregation engine  56 , each of which comprises software code executed by computing device  50 . Computing device  50  potentially will receive Terabytes (or more) of data from sensor  21  and other sensors and will store that data in storage device  70 . Filtering engine  55  will filter out (and potentially delete) all data that is not of interest. For example, it can filter out all IP addresses that are not of interest. This can be performed on a geographic basis, since IP addresses can be correlated with geography (by country or even by latitude-longitude coordinates) using services such as those offered by Akamai. So for example, if the data of interest is network activity in Canada, filtering engine  55  can filter out all data that is not associated with an IP address in Canada. Filtering engine  55  can perform filtering based on any available fields, including IP address, owner of IP address, geography, frequency of DNS lookup of an IP address, frequency of web communications involving an IP address, or key words within the network traffic (such as words often used in Internet scams, such as “prince”). In this manner, filtering engine  55  can substantially reduce the set of data stored in storage device  70 . 
         [0022]    Aggregation engine  56  can provide additional data related to the data received by computing device  50  or that is output by filtering engine  55 . For example, aggregation engine  56  can perform calculations, such as number of accesses to an IP address within a certain amount of time, minimum number of hits to an IP address, maximum number of hits to an IP address within a certain time period, the longest or shortest duration of a web session, or number of hits within the past X seconds. 
         [0023]    It will be understood that filtering engine  55  and aggregation engine  56  can be used in a customizable manner to distill data that is of interest to a customer of the operator of computing device  50 . For example, if the customer is interested in all accesses to an IP address associated with malware by IP addresses associated with the customer&#39;s network, then filtering engine  55  and aggregation engine  56  can output such a data set. 
         [0024]    With reference again to  FIG. 3 , computing device  50  can prepare reports for a particular customer. The reduced data set can then be sent to server  90 , optionally through queue  80 . Server  90  optionally is a web server that can interact with a customer over the Internet using known web server techniques and can be used to present the data of interest to the customer. 
         [0025]    Server  90  optionally can use the improved user interface technique discussed below. 
       User Interface 
       [0026]    With reference to  FIG. 5 , a typical prior art “backend” web server is depicted. Client computer  110  communicates with web server  130  over the Internet  120 . Web server  130  generates and serves web pages to client computer  110  using known web page techniques, such as HTML and HTTP. Web server  130  optionally obtains data from computing device  140 . Computing device  140  is coupled to data store  150 . Datastore  150  optionally stores raw data in a relational database, such as MySQL. Computing device  140  accesses data in datastore  150 . Web server  130  obtains data from computing device  140  and then displays that data using a variety of user interface techniques. 
         [0027]    An embodiment is now described with reference to  FIG. 6 . A screen shot of an improved user interface  210  is shown. This user interface  210  can be generated by computing devices such as web server  130  and can be viewed on computing devices such as client computer  110 . User interface  210  comprises menu bar  220  that in this example comprises the categories Geo, Product, Data Source, Organization, and Network. A user will select one of those categories through a click of the mouse or other known I/O techniques. 
         [0028]    In the example of  FIG. 6 , the Geo category  225  has been selected. User interface thereafter shows facet  230 , facet  240 , and facet  250 . Here, facet  230  corresponds to Regional Command, facet  240  corresponds to Country, and facet  250  corresponds to City. Facet  230 , facet  240 , and facet  250  can be altered by the user using a menu or other known interface that allows the user to select a field to use for a facet, such as continent, language, hemispheres, etc. These fields typically will correspond to the fields of data stored in datastore  150 . 
         [0029]    In the example of  FIG. 6 , the facets themselves depict data that is generated in real-time. For example, facet  230  can show different regional commands, each of which is next to a number. That number can represent data of interest, such as number of IP addresses used, number of computers accessing the Internet, etc. The contents of facets  230 ,  240 , and  250  are updated periodically. For example, they can be refreshed once every second. 
         [0030]    Notably, the user can generate a query by clicking within any of facets  230 ,  240 , and  250 . For example, by clicking on the “NORTHCOM” item in facet  230 , the countries displayed within facet  240  would change to display only the countries in NORTHCOM. However, a user also could select “Canada” within facet  240  from the outset. Facet  230  would then change to show “NORTHCOM” as selected, and facet  250  would update to display only cities within Canada. 
         [0031]    In addition, as a new data item is added to datastore  150 , the items within facets  230 ,  240 , and  250  will change. For example, if a city in Canada is added to datastore  150 , that city will be displayed in facet  250  when cities in Canada are being displayed. In this manner, facets  230 ,  240 , and  250  are dynamic and are updated in real-time. 
         [0032]    With reference now to  FIG. 7 , another aspect of user interface  210  is shown. Here, three items from facets  230 ,  240 , and  250  have been selected (in this example, URL, country, and application). The data is culled by computing device  140  and sent to web server  130 , and the results are displayed graphically on interactive map  260 . Interactive map  260  displays colored numbers for each result. If a user selects one of the colored numbers, additional data about that particular site is displayed. 
         [0033]    With reference to  FIG. 8 , another embodiment is shown. An exemplary screen shot of user interface  270  is shown. As with the embodiment of  FIG. 7 , user interface  270  includes an interactive map  280 . However, interactive map  280  displays the data dynamically. In this example, the size of the colored circles varies depending on the magnitude of the data being measured. In this example, the number of hits to an IP address are being tracked. For each IP address, the number of hits are measured over a period of time t 1 . The size of the circles displayed are directly proportional to the number of hits measured over time t 1  for that locale. This allows a user to watch interactive map  280  and quickly understand the change in usage patterns simply by watching the colored circles. 
         [0034]    An alternative embodiment is shown in  FIG. 9 .  FIG. 9  depicts an exemplary screen shot of user interface  290 . User interface  290  includes interactive map  300 . Interactive map also includes the numerical value of the data of interest (such as number of devices with IP addresses), and each colored circle optionally includes a pie chart indicating other data of interest (such as type of web server used; e.g., “Apache”). 
         [0035]    References to the present invention herein are not intended to limit the scope of any claim or claim term, but instead merely make reference to one or more features that may be covered by one or more of the claims. Materials, processes and numerical examples described above are exemplary only, and should not be deemed to limit the claims. It should be noted that, as used herein, the terms “over” and “on” both inclusively include “directly on” (no intermediate materials, elements or space disposed there between) and “indirectly on” (intermediate materials, elements or space disposed there between). Likewise, the term “adjacent” includes “directly adjacent” (no intermediate materials, elements or space disposed there between) and “indirectly adjacent” (intermediate materials, elements or space disposed there between). For example, forming an element “over a substrate” can include forming the element directly on the substrate with no intermediate materials/elements there between, as well as forming the element indirectly on the substrate with one or more intermediate materials/elements there between.