Patent Description:
In current network traffic processing systems, data streams comprise several events that contain data, also called data fragments, wherein the events are processed individually and independently of each other without any linkage between the events processed. Thus, no conclusion can be drawn whether the respective data of the several events is linked with each other or not, as the data is processed individually and independently of each other.

Moreover, the systems use a data model that is decoder specific so that each decoder type requires its own data model. In fact, the values of the data, namely the decoded values, are of semantic type. Therefore, it is not possible to link data decoded by different decoder types. Put another way, a single value decoded from the data stream does not have a defined relationship to another value. Thus, only the order of the values may be of importance.

In fact, the data is streamed continuously so that certain matching information may hit a rule engine too late, which in turn requires a memory consuming aggregation within the decoder. Furthermore, the aggregation may be done in the decoder only in parts. Thus, certain information might be missed due to the limited memory capacities.

To avoid this risk, the decoder that receives and processes the data streams has to keep all information gathered until the end of the respective data flow occurred. However, this concept still requires high memory capacities that increase the overall costs of the system.

<CIT> shows a synchronous packet-processing pipeline with several data-plane stateful processing units (DSPUs) that are arranged in a serial module chain, wherein the DSPUs process data packets received, wherein results of the respective processing is inserted into packet header vectors.

In <CIT>, a virtualized media processing pipeline is shown.

<CIT> shows a datapath pipeline with several stages located in series, wherein the respective stages process data packets received one after the other, wherein processed packets are outputted by the datapath pipeline.

Accordingly, there is a need for a simple way to analyze and interpret at least one data stream in a cost-efficient manner.

The invention provides a system for analyzing and interpreting at least one data stream. The system comprises a plurality of modules, at least one of which is a management module that comprises a user interface. The system comprises several data processing modules that are arranged in a serial module chain with an input and an output. The input of the module chain is configured to receive interception data. The system is configured to use a rule engine to structure the data in accordance with at least one matching rule such that the several data processing modules are configured to structure the interception data received via the input such that the output of the module chain is configured to output structured interception data. The serial module chain is configured to process data only in direction from the input to the output. The management module is configured to communicate with the data processing modules of the serial module chain. Each of the data processing modules is configured to announce information with regard to its own capability, namely capability information. The system is configured to forward the capability information of a single data processing module in the serial module chain in data flow direction for configuration of the data processing modules. Each data processing module is configured to automatically configure itself based on the capability information received from the at least one previous data processing module, thereby enabling an auto-configuration of the system.

Further, the invention provides a method of automatic configuration of a system for analyzing and interpreting at least one data stream, with the following steps:.

As the capability information of a single data processing module is forwarded to the next data processing modules in data flow direction, it is ensured that changing one of the several data processing modules does not affect the overall configuration of the system. In fact, the configuration of the system is adjusted automatically when a single data processing module or several data processing modules are exchanged or rather replaced. The auto-configuration is ensured by forwarding the capability information among each other.

According to an aspect, the capability information relates to semantic information and/or processing information. The data processing modules are enabled to retrieve data from the different kind of information.

According to another aspect, each data processing module is configured to adjust its capability based on received capability information. In fact, each data processing module automatically configures itself based on the capability information received from at least one previous data processing module.

Particularly, the management module is connected to the serial module chain. The management module is configured to communicate with the data processing modules of the serial module chain. Particularly, the management module receives information from the data processing modules and processes the respective information. In addition, the management module may control the data processing modules in a certain way based on the information received. Put another way, information concerning the data processing modules is gathered by the management module.

The management module may be connected to each data processing module of the serial module chain. Thus, the management module may gather information of each data processing module. The information may also concern log and health information that is analyzed for controlling purposes.

For instance, the management module is configured to receive capability information of each single data processing module in the serial module chain. The capability information is used for automatic configuration of the system, particularly the individual data processing modules.

Another aspect provides that the management module is configured to adjust the user interface based on the capability information received from the data processing modules of the serial module chain. The user interface may be adapted with regard to the respective data processing modules provided in the serial module chain. In other words, exchanging or rather replacing the data processing modules may have an influence on the functionality of the system which might be indicated on the user interface.

The management module may be configured to obtain log and health information from each data processing modules of the serial module chain. The log and health information provides further information concerning the individual data processing modules which is taken into account by the management module.

According to another aspect, the user interface is a graphical user interface. Thus, the respective information, for instance the information assigned to the capability of the individual data processing modules or rather the log and health information, may be outputted graphically.

Generally, the data fragments may correspond to application data fragments in the respective data stream.

Thus, the at least one decoder unit may be configured to decode different application protocols.

The data streams may be internet protocol data streams, for instance data streams assigned to Voice over Internet Protocol (VoIP), also called IP telephony, e-mail and/or hypertext transfer protocol (http).

The extraction module may receive different data streams, wherein tagged data fragments are outputted. The tagged data fragments including the structure data are forwarded to the aggregation module for aggregating the data fragments.

In fact, the decoder unit may define the unitary tagging data model or rather the unitary tagging data structure, wherein the unitary tagging data structure is announced by the decoder unit.

The unitary tagging data model or rather the unitary tagging data structure may generally relate to contexts, data fields and relations.

The unitary tagging data model/structure may comprise context identifications (context IDs) that are globally unique. This ensures that the context IDs can be assigned in an unambiguous manner.

The decoder unit of the extraction module may stream the different data fields as they become available.

The data fields originating from the same decoder unit may have a common root context, which identifies the processing decoder unit.

Accordingly, a system may be provided that is configured to look for certain information and to spit out everything related to it in a structured way. Hence, a system may be provided that is configured to retrieve data fragments, for instance metadata, content and/or raw data, from monitored data, namely data streams, related to at least one matching rule in a structured way. In fact, the system may use a rule engine to structure the data in accordance with the at least one matching rule.

Therefore, an event based data processing model, also called context data model, may be provided that is used by the system.

Accordingly, the system for analyzing and interpreting at least one data stream may correspond to a network traffic processing system.

Generally, all aspects described above apply for the system as well as the method in a similar manner.

<FIG> shows a system <NUM> that is configured to analyze and interpret at least one data stream, wherein the data stream processed by the system <NUM> comprises data, for instance metadata, raw data and/or content.

The system <NUM> comprises a plurality of modules <NUM>, namely a management module <NUM> as well as several data processing modules <NUM>.

The data processing modules <NUM> are arranged in a serial module chain <NUM> that has an input <NUM> and an output <NUM>.

The input <NUM> of the module chain <NUM> is configured to receive data of the data streams, also called interception data, whereas the output <NUM> of the module chain <NUM> is configured to output structured interception data.

Therefore, the several data processing modules <NUM> assigned to the serial module chain <NUM> are configured to structure the interception data received via the input <NUM>.

In the shown embodiment, the data processing modules <NUM> comprise at least one balance controller module <NUM>, at least one flow filter module <NUM>, at least one extraction module <NUM> as well as at least one aggregation module <NUM>.

These different modules <NUM> - <NUM> are connected in series with each other. Hence, the at least one extraction module <NUM> is directly connected with the at least one aggregation module <NUM>.

In fact, each kind of module <NUM> - <NUM> is provided several times defining a certain processing layer S1 - S4.

Moreover, at least one collection gateway module <NUM> as well as at least one converter module <NUM> may be provided that are located downstream and upstream of the serial module chain <NUM>, respectively.

The upstream located converter module <NUM> may receive non-Ethernet frames that are converted into Ethernet frames and Link metadata which is inputted in the serial module chain <NUM> together with the Ethernet frames and the Link metadata as shown in <FIG>.

The collection gateway module <NUM> receives aggregated data fragments from the aggregation module <NUM> as will be described later.

In fact, the at least one collection gateway module <NUM> as well as at least one converter module <NUM> may also be provided several times defining processing layers S0, S5.

For processing the data of the data streams, the extraction module <NUM> has at least one decoder unit <NUM> as well as a tagging unit <NUM>.

The decoder unit <NUM> is configured to decode data fragments of the data stream received, namely the interception data. In other words, the respective decoder unit <NUM> identifies and extracts data fragments from the data stream processed by the extraction module <NUM>.

The tagging unit <NUM> of the respective extraction module <NUM> is configured to tag the data fragments decoded based on a unitary tagging data model, also called unitary tagging data structure, that is schematically shown in <FIG>.

Accordingly, the extraction module <NUM> is configured to output extracted data fragments that were initially encompassed within the data stream. The extracted data fragments comprise a tag according to the unitary tagging data model/structure, which defines a relation between data fragments.

As shown in <FIG>, a plurality of decoder units <NUM> are provided that are generally configured to process different data streams. The plurality of decoder units <NUM> may be assigned to dedicated extraction modules <NUM>.

In turn, the tagging unit <NUM> is assigned to the plurality of decoder units <NUM> such that the tagging unit <NUM> is configured to tag data fragments of different data streams that were decoded by the plurality of decoder units <NUM> previously. Hence, the respective tags define a relation between data fragments originating from different data streams.

Accordingly, a single tagging unit <NUM> may be provided. Alternatively, one tagging unit <NUM> per extraction module <NUM> is provided.

In fact, the tagging unit <NUM> that uses the unitary tagging data model/structure is generally configured to apply this tagging data model to the data fragments of the different data streams decoded by the plurality of decoder units <NUM>.

If necessary, the unitary tagging data model may be adapted for every decoder unit <NUM> individually. This depends on the decoder unit <NUM> as well as the intended use. In fact, the tag may comprise information with regard to the respective decoder unit <NUM>, namely a decoder identification (decoder ID).

Generally, the data fragments may correspond to application data fragments that are processed by the system <NUM>. Thus, the at least one decoder unit <NUM> may be configured to decode different application protocols. Moreover, several decoder units <NUM> may be provided that are assigned to a certain (application) protocol.

As the extraction module <NUM> is located within the module chain <NUM>, the extraction module <NUM> has its own input <NUM> as well as its own output <NUM>.

The extraction module <NUM> receives via its input <NUM> different data streams, particularly Ethernet frames, that are processed by the extraction module <NUM> as described above, namely by using the at least one decoder unit <NUM> as well as the tagging unit <NUM>. Thus, the extraction module <NUM> outputs via its output <NUM> tagged data fragments for further processing, particularly metadata, content and frames.

As described above, the extraction module <NUM> is directly connected with the aggregation module <NUM>. Thus, the aggregation module <NUM> receives the tagged data fragments from the extraction module <NUM> for aggregation purposes.

Since the extraction module <NUM> applies the unitary tagging data model on the data fragments, at least one single data fragment ("DF") is outputted that includes structure data ("Str. "), as indicated in <FIG>, for instance. The structure data ("Str. ") may be assigned to the unitary tagging data model/structure applied, which is shown in <FIG> for an example concerning e-mail data.

In general, the extraction module <NUM> is configured to output a plurality of single data fragments ("DF") each including structure data ("Str. "), wherein relationships between the single data fragments are defined by the corresponding structure data.

As shown in <FIG>, the tagging data model/structure provides different types of relations, namely sibling relation as well as parent relation.

The sibling relation may correspond to different information that relates to an e-mail header such as addressee ("to"), sender ("from") and/or subject.

The parent relation may correspond to the relation of data fragments with regard to the hierarchy such as "Addressee ("to") - Header - Mail", "Sender ("from") - Header - Mail" and/or "Subject - Header - Mail".

Each of the above mentioned data fragments are assigned to values that may be of semantic type (addressee ("to"), sender ("from") and/or subject) and/or data type (string, number, Boolean and so on).

The unitary tagging data model/structure applied provides a structure that may put values assigned to data fields or rather data fragments in relation to each other, wherein the structure data/information corresponds to the above-mentioned context.

For instance, the unitary tagging data model/structure used comprises a unique identifier for a specific context identification (ID) and a type of context that defines how to interpret a data fragment associated with it, as shown in <FIG>.

In the shown example of <FIG>, the unitary tagging data model/structure is shown for e-mail data that comprises context (structure data) which relates to the mail, the header as well as the body, whereas the data fragments are assigned to data fields associated with the header or rather the body.

<FIG> shows that the respective structure of the data fragments can be described by vectors, namely context vector or rather context type vectors.

In the shown example of <FIG>, the data fragment/field with value "A" is described by the context vector [<NUM>, <NUM>, <NUM>] as well as the context type vector [<NUM>, <NUM>, <NUM>].

Therefore, each single data fragment has a corresponding structure data (context) wherein the structure data may be defined by the respective vectors.

As shown in <FIG>, the structure data corresponds to a data tree format or a data graph.

The unitary format used may be identical for different data streams processed by the system <NUM>.

The extraction module <NUM> that processes several data streams may output several data fragments with at least partially identical structure data within one message that is forwarded to the aggregation module <NUM>.

As shown in <FIG>, the data fragments assigned to the values "A", "B", "C" may have partially identical structure data as they are assigned to the same context vector [<NUM>, <NUM>, <NUM>, as well as the same context type vector [<NUM>, <NUM>, <NUM>], as the respective data fragments relate to header information, namely the same context.

<FIG> also reveals that a root context as well as a root context ID are provided, which refer to the first element of the respective vector. In fact, this indicates the respective instance.

The data fragments may be outputted by the extraction module <NUM> continuously such that the tagged data fragments are directly forwarded or rather streamed without any buffering.

Furthermore, a rule engine <NUM> may be provided, particularly integrated in the extraction module <NUM>.

The rule engine <NUM> forwards the (extracted) data fragments including the structure data to the aggregation module <NUM> for further processing if certain criteria will be fulfilled.

In fact, the rule engine <NUM> is configured to forward the data fragments including the structure data and input data stream, namely the raw data, to the aggregation module <NUM> for further processing if certain criteria will be fulfilled.

The aggregation module <NUM> is configured to receive the outputted data fragments from the extraction module <NUM>, namely the data fragments including the structure data as described above.

For further processing, the aggregation module <NUM> is configured to aggregate the data fragments received according to one predefined structure data for different protocols, data streams and/or decoder units <NUM> that are assigned to the at least one extraction module <NUM>.

This concept is schematically shown in <FIG> wherein four different data packages S1 to S4 are received by the extraction module <NUM> and processed appropriately such that data fragments ("DF") including structure data ("Str. ") are outputted by the extraction module <NUM> in an already structured manner.

The data fragments ("DF") including structure data ("Str. ") are forwarded to the aggregation module <NUM> for aggregation.

The aggregation module <NUM> is configured to aggregate the data fragments ("DF") including structure data ("Str. ") according to one predefined structure data as shown in <FIG>.

Hence, the structure data ("Str. ") of the data fragments ("DF") influence the aggregation done by the aggregation module <NUM>.

The predefined structure data may relate to different protocols, data streams and/or decoder units <NUM> assigned to the at least one extraction module <NUM>.

The aggregation module <NUM> comprises at least one output <NUM> via which the aggregated data fragments, namely the data fragments including structure data processed by the aggregation module <NUM>, are outputted as shown in <FIG> and <FIG>. Hence, aggregated metadata, content and/or frames are outputted.

Generally, the aggregation module <NUM> may have several outputs such that different aggregated data fragments may be outputted via the outputs <NUM> wherein the output data comprise at least in parts identical data.

The aggregated data fragments are typically outputted together with corresponding raw data from the respective data stream inputted in the extraction module <NUM>. Hence, the data of the data streams is outputted by the aggregation module <NUM>, namely the respective data fields ("DF") or rather their values as shown in <FIG>.

In general, the aggregation module <NUM> may convert the data fragments received into a different (application) protocol with respect to the receiving (application) protocol. This may simplify further processing of the aggregated data fragments, for instance by the collection gateway module <NUM>.

The receiving protocol used for communication between the extraction module <NUM> and the aggregation module <NUM> may generally be compatible to at least two different types of decoder units <NUM>.

Thus, at least two different types of decoder units <NUM> may be used for processing different data streams wherein the decoded data fragments are forwarded to the aggregation module <NUM>.

Hence, the aggregation module <NUM> is configured to receive the tagged data fragments from at least two different types of decoder units <NUM> that may be assigned to at least two different extraction modules <NUM>.

Accordingly, the tagged data fragments may relate to the unitary data model/structure even though they were identified and extracted by different extraction modules <NUM>, particularly different types of decoder units <NUM>.

Hence, the different data streams may be processed by different extraction modules <NUM>, wherein the aggregation module <NUM> is configured to combine the tagged data fragments received from the different extraction modules <NUM> to one single session data that is outputted.

The aggregation module <NUM> is configured to output the single session data that encompasses the tagged data fragments extracted and tagged by the at least two different extraction modules <NUM>.

In general, each extraction module <NUM> processes the data streams in a manner as described above, wherein the unitary data model/structure is applied that might be adapted with regard to the decoder identification (decoder ID) as already mentioned, which identifies the respective decoder unit <NUM> in a unique manner.

Anyway, multi data flows, namely different data streams, that relate to different (application) protocols can be processed by the different extraction modules <NUM> or rather the decoder units <NUM>, which ensures a higher scalability since the information gathered, namely the data, is distributed among the extraction modules <NUM> or rather the decoder units <NUM>. The scalability can be improved up to Tbit/s.

For instance, Sessions Initiation Protocol (SIP) data packets, namely a first data stream, may arrive which are processed by the at least one extraction module <NUM>. Respective correlation information concerning the data stream arrived may be broadcasted among different extraction modules <NUM>, namely within the processing layer S3.

For this purpose, the respective decoder units <NUM> of the at least one extraction module <NUM> may share relevant correlation information, for instance via broadcast mechanism.

Then, Real-Time Transport Protocol (RTP) data packets, namely a second data stream, may arrive that are also processed by the at least one extraction module <NUM>, namely within the processing layer S3.

The rule engine <NUM> may relate to forwarding context if RTP data packets and SIP data packets arrive.

If the rule provided by the rule engine <NUM>, namely the respective criteria, is fulfilled, at least the context is forwarded to the subsequent aggregation module <NUM>, namely processing layer S4.

Irrespective of the number of decoder units <NUM> or rather extraction modules <NUM>, the aggregation module <NUM> is configured to aggregate the data fragments received for a predefined time.

<FIG> shows that the data fragments including the structure data are assigned to context information which groups all information together.

For instance, the context information encompasses the decoder identification (decoder ID), the context vector, the context type vector, the data field type, the data field value as well as the rule ID and the mandate ID, which trigger the forwarding of the data field from the extraction module <NUM> to the aggregation module <NUM>. As already mentioned above, the context vector as well as the context type vector relate to the structure data that provides relative information with regard to the respective data fragment.

The aggregation module <NUM> is configured to apply an aggregation function on the context information, wherein the aggregation function comprises aggregation function parameters as illustrated in <FIG>.

As shown, the aggregation function parameters relate to the decoder identification (decoder ID), the context type vector prefix as well as the aggregation timeout relating to the predefined time. Hence, the aggregation function parameters relate to a tuple of values as indicated above.

The aggregation module <NUM> is configured to create at least one pile for a matching parameter set while processing the aggregation function as well as the context information.

In other words, the matching condition of the aggregation function is assigned to the decoder ID and the context type vector prefix (CtxTVP) being part of the context type vector (CtxTV).

Thus, parameters are assigned to a single pile that have matching parameters, particularly matching decoder identification and context type vector prefix.

As also shown in <FIG>, the respective pile comprises at least one data field with common context vector prefix. Moreover, secondary data like rule ID and/or mandate ID may be assigned to the respective pile.

Thus, the aggregation module <NUM> is configured to output aggregated data fragments that are structured, namely by piles that relate to the respective structure data such as context type vector and context vector.

<FIG> shows an example for aggregation wherein a decoder type corresponds to a "Mail-Decoder", the context type vector prefix is equal to "Mail" and the aggregation timeout corresponds to "<NUM> seconds".

As shown, the data fields assigned to "<NUM>", "<NUM>" and "<NUM>" are aggregated, as the data field indicated with "<NUM>" has a different context vector prefix.

According to another example (not shown), the context type vector prefix may be equal to "Mail - Header". Then, the data fields assigned to "<NUM>" and "<NUM>" would only be aggregated, as the context vector of data field labelled with "<NUM>" is assigned to the context vector [<NUM>, <NUM>] corresponding to the context type vector prefix "Mail - Body".

Apart from the data processing modules <NUM> and their functionality described in detail above, the system <NUM> comprises the management module <NUM>.

The serial module chain <NUM> processes the data only in direction from its input <NUM> to its output <NUM>.

Each of the data processing modules <NUM> in the serial module chain <NUM> provides capability information that is used in data flow direction for configuring the following data processing modules <NUM>. In other words, each of the data processing modules <NUM> announces information with regard to its own capability. The respective information is forwarded/distributed in data flow direction such that subsequent data processing modules <NUM> obtain the respective information.

Each data processing module <NUM> that receives the respective capability information of one of the previous data processing modules <NUM> is configured to adjust its own capability based on the capability information received from the previous one.

Generally, the respective capability information may relate to semantic information and/or processing information.

In fact, replacing one of the several data processing modules <NUM> within the serial module chain <NUM> does not require to reconfigure the whole system <NUM> manually since the respective capability information of the new several data processing module(s) <NUM> is distributed to the subsequent data processing modules <NUM> effectively.

Hence, an automatic reconfiguration of the system <NUM> is achieved, as the system <NUM> is modular.

This means that the data processing modules <NUM> can be interchanged or rather replaced without the need for manual reconfiguration, as the reconfiguration of the system <NUM> is automatically done by the data processing modules <NUM> themselves announcing their respective capability information.

The respective capability information announced is received by the subsequent data processing modules <NUM> that adapt their settings appropriately with regard to the capability information announced such that a reconfiguration of the whole system <NUM> takes place.

As the management module <NUM> is connected to the serial module chain <NUM>, in particular each of the several data processing modules <NUM>, the management module <NUM> also receives the respective capability information of each data processing module <NUM>.

The management module <NUM> has a graphical user interface (GUI), wherein the management module <NUM> is enabled to adjust the respective graphical user interface based on the capability information received from the data processing module(s) <NUM>.

Accordingly, the user interface is adapted to the current capability of the whole system <NUM>.

Furthermore, the management module <NUM> may receive log and health information from the several data processing module(s) <NUM> that may be processed by the management module <NUM>.

In addition, the management module <NUM> controls the several data processing module(s) <NUM> based on the log and health information and/or the capability information received from the data processing module(s) <NUM>.

Claim 1:
A system (<NUM>) for analyzing and interpreting at least one data stream, wherein the system (<NUM>) comprises a plurality of modules (<NUM>), at least one of which is a management module (<NUM>) that comprises a user interface, wherein the system (<NUM>) comprises several data processing modules (<NUM>) that are arranged in a serial module chain (<NUM>) with an input (<NUM>) and an output (<NUM>), wherein the input (<NUM>) of the module chain (<NUM>) is configured to receive interception data, wherein the system (<NUM>) is configured to use a rule engine to structure the data in accordance with at least one matching rule such that the several data processing modules (<NUM>) are configured to structure the interception data received via the input (<NUM>) such that the output (<NUM>) of the module chain (<NUM>) is configured to output structured interception data, wherein the serial module chain (<NUM>) is configured to process data only in direction from the input (<NUM>) to the output (<NUM>), characterized in that the management module (<NUM>) is configured to communicate with the data processing modules (<NUM>) of the serial module chain (<NUM>), wherein each of the data processing modules (<NUM>) is configured to announce information with regard to its own capability, namely capability information, wherein the system (<NUM>) is configured to forward the capability information of a single data processing module (<NUM>) in the serial module chain (<NUM>) in data flow direction for configuration of the data processing modules (<NUM>), wherein each data processing module is configured to adjust its capability based on the received capability information such that each data processing module (<NUM>) is configured to automatically configure itself based on the capability information received from the at least one previous data processing module (<NUM>), thereby enabling an auto-configuration of the system (<NUM>).