Patent Document ID: 20170208077
Application ID: 15404623
Patent Flag: 0

Claim One:
1. A method for a clustered architecture real-time network traffic capture and analysis, comprising: operating a kentik data engine (KDE) on clustered computer server hardware; transmitting network flow and BGP data to the KDE from Agent software operating on a virtual machine within the computer network, or directly from router devices within the computer network, or both from Agent software and directly from router devices; listening, by the Agent software, to UDP traffic on the computer network; querying, by the Agent software, the KDE for device information identified in the UDP traffic; forking a separate Agent process for each unique device detected in the UDP traffic; transforming, by the separate Agent process for a unique device, UDP traffic for the unique device into the one or more Kflow records; launching, by the separate Agent process for the unique device, a BGP daemon for the unique device; polling, by the Agent process, network devices on the computer network for simple network management protocol (SNMP) data; merging, by the Agent process, BGP data collected by the BGP daemon and SNMP data into the one or more Kflow records; posting the one or more Kflow records over the internet from the Agent software to a virtual internet protocol address (VIP) of the KDE; transmitting, from a router device within the computer network, UDP data to a virtual internet protocol address (VIP) of the KDE; initiating, by the KDE, a KDE child process for the router device; transmitting, from the router device to the KDE child process, BGP data; gathering, by the KDE child process, domain name service (DNS) data and simple network management protocol (SNMP) data of the router device; integrating, by the KDE child process, the UDP, BGP, DNS, and SNMP data into the one or more Kflow records; compiling a preconfigured interval of Kflow records into a FastKFlow record, and storing the FastKFLow record for historical analysis; indexing metadata about Kflow records into a metadata system (MDS) based on a time interval, a device identifier, and a local server server name. receiving a SQL query; breaking the SQL query into one or more subqueries such that there is one subquery per device per time interval covered by the SQL query; identifying, through the MDS, one or more storage servers containing data records for the particular device and time interval of each subquery; sending each subquery to an identified, available, and least loaded storage server; checking, by each storage server, for relevant subquery results in a server cache, and, if not in the server cache, queuing the subquery for retrieving results from disk; updating, by each storage server, the storage server cache for retrieved results and copying the cached results to any additional nodes in the clustered computer server hardware; processing Kflow records into an alpha flow of data tuples comprising at least a source IP address, a destination IP address, a source port, a destination port, a protocol, a number of bytes, and a set of packet data; processing the alpha flow into a beta flow of key value pairs; comparing beta flow to one or more trigger rules for alert conditions based on values for given keys in the trigger rules; processing a top set of keys by value for retention; writing a history of retained beta flow to a history database on a large time period interval of one hour or longer; and identifying deviations in the beta flow values with values in the history database for alert from historical usage patterns separately from trigger rules.