Patent Document ID: 9942253
Application ID: 15404623
Patent Flag: 1

Claim One:
1. A method for a clustered architecture real-time network traffic capture and analysis, comprising: operating a Kentik Data Engine (KDE) on clustered computer server hardware; transmitting network flow and Border Gateway Protocol (BGP) data to the KDE from Agent software operating on a virtual machine within the computer network, or directly from router devices within the computer network, or both from Agent software and directly from router devices; listening, by the Agent software, to User Datagram Protocol (UDP) traffic on the computer network; querying, by the Agent software, the KDE for device information identified in the UDP traffic; forking a separate Agent process for each unique device detected in the UDP traffic; transforming, by the separate Agent process for a unique device, UDP traffic for the unique device into the one or more Kflow records, each Kflow record associating one or more templates with one or more network flow data values; launching, by the separate Agent process for the unique device, a BGP daemon for the unique device; polling, by the Agent process, network devices on the computer network for Simple Network Management Protocol (SNMP) data; merging, by the Agent process, BGP data collected by the BGP daemon and SNMP data into the one or more Kflow records; posting the one or more Kflow records over the internet from the Agent software to a Virtual Internet Protocol (VIP) address of the KDE; transmitting, from a router device within the computer network, UDP data to the VIP of the KDE; initiating, by the KDE, a KDE child process for the router device; transmitting, from the router device to the KDE child process, BGP data; gathering, by the KDE child process, domain name service (DNS) data and SNMP data of the router device; integrating, by the KDE child process, the UDP, BGP, DNS, and SNMP data into the one or more Kflow records; compiling a preconfigured interval of Kflow records to form a FastKFlow record, and storing the FastKFlow record for historical analysis; indexing metadata about Kflow records into a metadata system (MDS) based on a time interval, a device identifier, and a local server server name; receiving a Structured Query Language (SQL) query; breaking the SQL query into one or more subqueries such that there is one subquery per device per time interval covered by the SQL query; identifying, through the MDS, one or more storage servers containing data records for the particular device and time interval of each subquery; sending each subquery to an identified, available, and least loaded storage server; checking, by each storage server, for relevant subquery results in a server cache, and, if not in the server cache, queuing the subquery for retrieving results from disk; updating, by each storage server, the storage server cache for retrieved results and copying the cached results to any additional nodes in the clustered computer server hardware; processing Kflow records into an alpha flow of data tuples comprising at least a source IP address, a destination IP address, a source port, a destination port, a protocol, a number of bytes, and a set of packet data; processing the alpha flow into a beta flow of key value pairs; comparing beta flow to one or more trigger rules for alert conditions based on values for given keys in the trigger rules; processing a top set of keys by value for retention; writing a history of retained beta flows to a history database on a large time period interval of one hour or longer; and identifying deviations in beta flow values of the retained beta flows with values in the history database for alerts from historical usage patterns separately from trigger rules.