Abstract:
Network traffic is distinguished at a conversation level, providing sampling decision capability. A hash value is determined based on IP addresses and protocol type, giving unique identifiers for individual conversations.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    This invention relates to networking, and more particularly to distinguishing and sampling network traffic at a conversation level. 
         [0002]    Application performance management (APM) uses monitoring and/or troubleshooting tools for observation of network traffic and for network and application optimization and maintenance. In high traffic networks, data volume can lead to oversubscription—the condition where the incoming data rate is too high for network/application monitoring systems to process. One way this problem manifests itself is in terms of analysis latency. There is software latency in all application specific application analyzers (applications such as: Http, Oracle, Citrix, TCP, etc). When it is attempted to analyze too much data, the aggregate latency across various discrete portions of a monitoring system puts enough collective drag on the overall system that it becomes difficult to keep up with processing and analyzing the incoming data. It is computationally impractical to perform full analysis in real time of every packet/flow/conversation on a highly utilized computer network. 
       SUMMARY OF THE INVENTION 
       [0003]    An object of the invention is to provide for packet characterization and sample selection based on socket connections and conversations, addressing the problem of not being able to generate accurate performance measurements when traffic rates exceed the measurement device capability. 
         [0004]    Accordingly, it is another object of the present invention to provide an improved network data sampling. 
         [0005]    It is a further object of the present invention to provide an improved network monitoring system that distinguishes network traffic at a conversation level. 
         [0006]    It is yet another object of the present invention to provide improved methods of network monitoring and analysis that enable improved distinguishing and sampling of network traffic at a conversation level. 
         [0007]    The subject matter of the present invention is particularly pointed out and distinctly claimed in the concluding portion of this specification. However, both the organization and method of operation, together with further advantages and objects thereof, may best be understood by reference to the following description taken in connection with accompanying drawings wherein like reference characters refer to like elements. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]      FIG. 1  is a block diagram of a network with a network analysis product interfaced therewith; 
           [0009]      FIG. 2  is a block diagram of a monitor device for distinguishing and sampling bi-directional network traffic at a conversation level; and 
           [0010]      FIG. 3  is a flow chart of an example determination of a conversation identifier. 
       
    
    
     DETAILED DESCRIPTION 
       [0011]    The system according to a preferred embodiment of the present invention comprises a monitoring system and method and an analysis system and method for distinguishing and sampling bi-directional network traffic at a conversation level. 
         [0012]    Referring to  FIG. 1 , a block diagram of a network with an apparatus in accordance with the disclosure herein, a network may comprise plural network clients  10 ,  10 ′, etc., which communicate over a network  12  by sending and receiving network traffic  14  via interaction with server  20 . The traffic may be sent in packet form, with varying protocols and formatting thereof. 
         [0013]    A network analysis device  16  is also connected to the network, and may include a user interface  18  that enables a user to interact with the network analysis device to operate the analysis device and obtain data therefrom, whether at the location of installation or remotely from the physical location of the analysis product network attachment. 
         [0014]    The network analysis device comprises hardware and software, CPU, memory, interfaces and the like to operate to connect to and monitor traffic on the network, as well as performing various testing and measurement operations, transmitting and receiving data and the like. When remote, the network analysis device typically is operated by running on a computer or workstation interfaced with the network. One or more monitoring devices may be operating at various locations on the network, providing measurement data at the various locations, which may be forwarded and/or stored for analysis. 
         [0015]    The analysis device comprises an analysis engine  22  which receives the packet network data and interfaces with data store  24 . 
         [0016]      FIG. 2  is a block diagram of a test instrument/analyzer  26  via which the invention can be implemented, wherein the instrument may include network interfaces  28  which attach the device to a network  12  via multiple ports, one or more processors  30  for operating the instrument, memory such as RAM/ROM  32  or persistent storage  34 , display  36 , user input devices (such as, for example, keyboard, mouse or other pointing devices, touch screen, etc.), power supply  40  which may include battery or AC power supplies, other interface  42  which attaches the device to a network or other external devices (storage, other computer, etc.). 
         [0017]    In operation, the network test instrument is attached to the network, and observes transmissions on the network to collect data and analyze and produce statistics and metadata thereon. Distinguishing and sampling of the network data at a conversation level enables coherent and consistent grouping and analysis of data and selection and differentiation of data of interest while allowing data not of interest to be ignored. 
         [0018]    To accomplish the distinguishing and sampling, a one way hash is made on conversations between two hosts based on identifying factors in the observed data, for example, a client IP address, the server IP address and the communication protocol being used between the two hosts. Each host has a unique IP address, and the client/server pair communicate with each other using a specific network protocol. The one way hash allows generating a unique identifier for a particular conversation. A conversation can then be sampled if it is a conversation of interest, and processed or stored based on the hash value. This allows a quick way to distinguish traffic at a conversation level, enabling decisions of whether the data is of interest and further processing to be quickly made. 
         [0019]      FIG. 3  is a flow chart of the process, wherein an observed packet exchanged between a client and server is selected at  44 , and the value of the hash function is initialized at block  46 . Next, the hash value is calculated at block  48 ,  50 ,  52  using the client IP address, the server IP address and the protocol type. The determined hash value may then be employed as a unique identifier as to the conversation to which this packet belongs. The example of  FIG. 3  illustrates a case with 3 separate calculations of hash based on client address, server address and protocol type, as an example and for convenience of illustration. The particular hash function and the manner of implementing the hash function can result in the hash keys being combined in a single step or multiple steps. 
         [0020]    As an example, in a particular embodiment, the hash function is chosen to be a CRC (cyclic redundancy check) function, crc32, applied to the source IP, destination IP and protocol type fields of a data packet. The hash determination may be performed by a network test instrument in accordance with the disclosure herein, by specific hash calculation hardware, or by software, in according with the speed required in the particular embodiment and operation environment. 
         [0021]    An example of using crc32 with accommodation of both IPV4 (32 bit) and IPV6 (128 bit) addresses is provided by the following, where client-IP-address[ ] is an array of 32 bit IP address values is given below. The example assumes that the address type is either IPV4 or IPV6, initially calculating the hash on the first 32 bits of address, and, if the address is IPV6 type, then calculating the hash using the additional 96 bits of the address: 
         [0000]    
       
         
               
               
             
               
               
             
               
               
             
               
               
             
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 hash = crc32(client-IP-address[0]); 
               
               
                   
                 if client-address-type == IPV6 
               
               
                   
                 { 
               
             
          
           
               
                   
                 hash = crc32(client-IP-address[1], hash); 
               
               
                   
                 hash = crc32(client-IP-address[2], hash); 
               
               
                   
                 hash = crc32(client-IP-address[3], hash); 
               
             
          
           
               
                   
                 } 
               
               
                   
                 hash = crc32(server-IP-address[0], hash); 
               
               
                   
                 if server-address-type == IPV6 
               
               
                   
                 { 
               
             
          
           
               
                   
                 hash = crc32(server-IP-address[1], hash); 
               
               
                   
                 hash = crc32(server-IP-address[2], hash); 
               
               
                   
                 hash = crc32(server-IP-address[3], hash); 
               
             
          
           
               
                   
                 } 
               
               
                   
                 hash = crc32(protocol-type, hash) 
               
               
                   
                   
               
             
          
         
       
     
         [0022]    The resulting hash value is then provides a unique way to identify the conversation as to protocol type and the sender and receiver. 
         [0023]    In the particular example above, a CRC function is employed, but other one way hash functions may be employed. In this way, the likelihood of a hash collision between multiple hosts/protocols is minimized. Accordingly, conversations between unique client/server pairs on a computer network can be accurately tracked. 
         [0024]    The invention provides ability to aggregate network packets into higher level constructs (conversations) that are uniquely identified by a generated ID derived from client/server IP addresses and the relevant network protocol used between them. Traffic sampling and hence, dynamic scaling, is then possible based on the conversation rather than individual packets. Other portions or combinations of portions of the conversation can be used for distinguishing and sampling, in addition to the IP addresses/protocol type example illustrated herein. 
         [0025]    While a preferred embodiment of the present invention has been shown and described, it will be apparent to those skilled in the art that many changes and modifications may be made without departing from the invention in its broader aspects. The appended claims are therefore intended to cover all such changes and modifications as fall within the true spirit and scope of the invention.