Patent Description:
Unfortunately, the IPFIX protocol may give rise to certain challenges for network equipment manufacturers. For example, to export the IPFIX data, the IPFIX protocol may necessitate and/or rely on certain lookups and/or translations. These IPFIX lookups and/or translations may be very resource intensive and/or demanding. As a result, these IPFIX lookups and/or translations may lead to performance issues. In traditional IPFIX technologies, these IPFIX lookups and/or translations may be performed by the observation domain.

In addition to being very resource intensive and/or demanding, these IPFIX lookups and/or translations may necessitate and/or rely on a complete Forwarding Information Base (FIB). In some examples, a complete FIB may include and/or represent a high number of routes (e.g., one million routes, two million routes, etc.), which collectively consume significant memory resources in the observation domain. As a result, to support the IPFIX protocol in traditional technologies, the observation domain may be limited to implementation on expensive physical devices that include sufficient memory resources.

Although the observation domain may reduce redundancy in the exported IPFIX data by deduplicating commonalities found in flows encountered across the observation points, traditional IPFIX technologies may fail to address and/or account for additional commonalities found across sets of observation domains. These traditional IPFIX technologies may thus be losing a certain amount of efficiency by failing to address and/or account for the additional commonalities found across sets of observation domains. The instant disclosure, therefore, identifies and addresses a need for additional and improved systems and methods for offloading IPFIX lookup and translation operations from observation domains.

<CIT> relates to a distributed network flow exporter.

<CIT> relates to aggregating flows by endpoint category.

According to a first aspect there is provided a method comprising: receiving at an Internet Protocol Flow Information Export "IPFIX" collector, at least one IPFIX message from an IPFIX exporter implemented on a remote device, the remote device comprising a router; identifying, within the IPFIX message, a data set exported by the IPFIX exporter implemented on the remote device; identifying, within the IPFIX message, a data-level indicator that indicates whether the data set is: a primary data set observed by an observation domain implemented on one or more line cards installed to a routing engine of the remote device; or a secondary data set derived by an observation cloud implemented on the routing engine of the remote device, wherein the derived data is obtained through lookup and/or translation and the observed data is obtained without lookup or translation; identifying, at the IPFIX collector, a database of a plurality of databases that corresponds to the data-level indicator identified within the IPFIX message, wherein the plurality of databases comprises a first database dedicated to storing primary data sets and a second database dedicated to storing secondary data sets; storing the data set in the database identified in accordance with the data-level indicator; and performing at least one action based at least in part on the data set stored in the identified database.

According to a second aspect there is provided a system comprising: at least one memory device that stores an Internet Protocol Flow Information Export "IPFIX" collector comprising a collector module, an identification module, a storage module, and an action module, wherein: the collector module of the IPFIX collector receives at least one IPFIX message from an IPFIX exporter implemented on a remote device, the remote device comprising a router; the identification module of the IPFIX collector: identifies, within the IPFIX message, a data set exported by the IPFIX exporter implemented on the remote device; identifies, within the IPFIX message, a data-level indicator that indicates whether the data set is: a primary data set observed by an observation domain implemented on one or more line cards installed to a routing engine of the remote device; or a secondary data set derived by an observation cloud implemented on the routing engine of the remote device, wherein the derived data is obtained through lookup and/or translation and the observed data is obtained without lookup or translation; identifies, at the IPFIX collector, a database of a plurality of databases that corresponds to the data-level indicator identified within the IPFIX message, wherein the plurality of databases comprises a first database dedicated to storing primary data sets and a second database dedicated to storing secondary data sets; the storage module of the IPFIX collector stores the data set in the database identified in accordance with the data-level indicator; the action module performs at least one action based at least in part on the data set stored in the identified database; and at least one physical processor configured to execute the collector module, the identification module, the storage module, and the action module.

According to a third aspect there is provided a computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to: receive, by an Internet Protocol Flow Information Export "IPFIX" collector, at least one IPFIX message from an IPFIX exporter implemented on a remote device, the remote device comprising a router; identify, within the IPFIX message, a data set exported by the IPFIX exporter implemented on the remote device; identify, within the IPFIX message, a data-level indicator that indicates whether the data set is: a primary data set observed by an observation domain implemented on one or more line cards installed to a routing engine of the remote device; or a secondary data set derived by an observation cloud implemented on the routing engine of the remote device, wherein the derived data is obtained through lookup and/or translation and the observed data is obtained without lookup or translation; identify, at the IPFIX collector, a database of a plurality of databases that corresponds to the data-level indicator identified within the IPFIX message, wherein the plurality of databases comprises a first database dedicated to storing primary data sets and a second database dedicated to storing secondary data sets; store the data set in the database identified in accordance with the data-level indicator; and perform at least one action based at least in part on the data set stored in the identified database.

Features from any of the above-mentioned embodiments may be used in combination with one another in accordance with the general principles described herein. These and other embodiments, features, and advantages will be more fully understood upon reading the following detailed description in conjunction with the accompanying drawings and claims.

Throughout the drawings, identical reference characters and descriptions indicate similar, but not necessarily identical, elements. While the example embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, the example embodiments described herein are not intended to be limited to the particular forms disclosed. Rather, the instant disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims.

The present disclosure describes various systems and methods for offloading IPFIX lookup and translation operations from observation domains. As will be explained in greater detail below, embodiments of the instant disclosure may implement an observation cloud that supports the IPFIX protocol. In some examples, the observation cloud may include and/or represent a set of observation domains each with various observation points. In such examples, the duties of exporting data to an IPFIX collector may be divided among the observation domains and the observation cloud in such a way that the observation domains need not perform lookup or translation operations in connection with IPFIX data exports.

Since, in these examples, the observation domains need not perform lookup or translation operations in connection with IPFIX data exports, the observation domain may be able to support the IPFIX protocol without a complete FIB. As a result, network equipment manufacturers may design and/or provide IPFIX-supported devices that are able to implement observation domains with less memory capacity than traditional IPFIX technologies.

A router includes a set of line cards installed to a routing engine. The line cards implement a set of observation domains, and the routing engine implemenst an observation cloud that interfaces with the observation domains. To achieve a workable division of duties for exporting data to an IPFIX collector, each observation domain may discover and/or export observable properties of various flows encountered by that domain's observation points. Such observable properties of the flows may include and/or represent any metadata, attributes, characteristics, and/or fields that are discoverable by simple inspections of the flows' packets. Accordingly, each observation domain may be responsible for exporting the observable properties of the flows to the IPFIX collector.

For the purposes of the present disclosure, the observable properties of the flows may be distinguished from derivable properties of the flows. The derivable properties may necessitate a lookup and/or a translation, whereas the observable properties may be discovered without a lookup or a translation. In this example, the observation cloud, as opposed to the observation domains, may discover and/or export the derivable properties of the flows encountered across the observation points of the various observation domains. Such derivable properties of the flows may include and/or represent any metadata, attributes, characteristics, and/or fields that necessitate one or more lookup and/or translation operations for discovery. Accordingly, the observation cloud may be responsible for exporting the derivable properties of the flows to the IPFIX collector.

In this example, a remote device may implement the IPFIX collector, which receives the IPFIX exports from the observation cloud and the various observation domains. At the remote device, the IPFIX collector may distinguish between the IPFIX exports originating from the observation cloud and the IPFIX exports originating from the observation domains. The IPFIX collector may thus store the IPFIX exports from the observation cloud in one database and the IPFIX exports from the observation domains in another database. The IPFIX collector may then unify, combine, and/or join the IPFIX exports from the one database with the corresponding IPFIX exports from the other database based at least in part on one or more common identifiers shared by those IPFIX exports. The resulting unification of IPFIX exports may include and/or represent a complete set of IPFIX data for the flows in question.

In this example, the IPFIX collector may perform and/or facilitate certain actions and/or services based at least in part on the unification of IPFIX exports. For example, the complete set of IPFIX data for a specific flow may be used by the IPFIX collector and/or another device or component to facilitate certain network services, such as traffic metering, traffic profiling, security evaluations, intrusion detection, accounting, and/or billing, among others.

The following will provide, with reference to <FIG>, <FIG>, and <FIG> detailed descriptions of example systems and corresponding implementations for offloading IPFIX lookup and translation operations from observation domains. Detailed descriptions of example IPFIX messages, primary data sets, secondary data sets, and/or unified data records will be provided in connection with <FIG>, <FIG>, <FIG>, and <FIG>, respectively. Detailed descriptions of computer-implemented methods for offloading IPFIX lookup and translation operations from observation domains will be provided in connection with <FIG> and <FIG>. In addition, detailed descriptions of an example computing system for carrying out these methods will be provided in connection with <FIG>.

<FIG> shows an example system <NUM> that facilitates offloading IPFIX lookup and translation operations from observation domains. As illustrated in <FIG>, system <NUM> may include one or more modules <NUM> for performing one or more tasks. As will be explained in greater detail below, modules <NUM> may include a collector module <NUM>, an identification module <NUM>, a storage module <NUM>, an action module <NUM>, an exporter module <NUM>, and/or a union module <NUM>. Although illustrated as separate elements, one or more of modules <NUM> in <FIG> may represent portions of a single module, application, and/or operating system.

In certain embodiments, one or more of modules <NUM> in <FIG> may represent one or more software applications or programs that, when executed by a processor of a computing device, cause the computing device to perform one or more tasks. For example, and as will be described in greater detail below, one or more of modules <NUM> may represent modules stored and configured to run on one or more computing devices, such as the devices illustrated in <FIG> (e.g., computing device <NUM>, computing device <NUM>, computing device <NUM>, and/or network device <NUM>) and/or the devices in <FIG> (e.g., exporter <NUM> and/or collector <NUM>). One or more of modules <NUM> in <FIG> may also represent all or portions of one or more special-purpose computers configured to perform one or more tasks.

As illustrated in <FIG>, example system <NUM> may also include one or more memory devices, such as memory <NUM>. Memory <NUM> generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and/or computer-readable instructions. In one example, memory <NUM> may store, load, and/or maintain one or more of modules <NUM>. Examples of memory <NUM> include, without limitation, Random Access Memory (RAM), Read Only Memory (ROM), flash memory, Hard Disk Drives (HDDs), Solid-State Drives (SSDs), optical disk drives, caches, variations or combinations of one or more of the same, and/or any other suitable storage memory.

As illustrated in <FIG>, example system <NUM> may also include one or more physical processors, such as physical processor <NUM>. Physical processor <NUM> generally represents any type or form of hardware-implemented processing device capable of interpreting and/or executing computer-readable instructions. In one example, physical processor <NUM> may access and/or modify one or more of modules <NUM> stored in memory <NUM>. Additionally or alternatively, physical processor <NUM> may execute one or more of modules <NUM> to facilitate offloading IPFIX lookup and translation operations from observation domains. Examples of physical processor <NUM> include, without limitation, Central Processing Units (CPUs), microprocessors, microcontrollers, Field-Programmable Gate Arrays (FPGAs) that implement softcore processors, Application-Specific Integrated Circuits (ASICs), portions of one or more of the same, variations or combinations of one or more of the same, and/or any other suitable physical processor.

As illustrated in <FIG>, example system <NUM> may further include one or more data sets, such as primary data set <NUM> and/or secondary data set <NUM>. In some examples, primary data set <NUM> may include and/or represent data observed, sampled, and/or identified by the data path, data plane, and/or forwarding plane of a network device. In one example, primary data set <NUM> may include, represent, and/or identify one or more common properties and/or elements of flows monitored across observation points of an observation domain. Examples of data, common properties, and/or elements incorporated in primary data set <NUM> include, without limitation, ingress interface indexes, next hop indexes, flow directions, forwarding statuses, data link frame sizes, data link frame selections, gateways, outgoing interfaces (OIFs), combinations or variations of one or more of the same, and/or any other suitable data, common properties, and/or elements.

In some examples, primary data set <NUM> may include, specify, and/or identify one or more common properties identifiers. In such examples, these common properties identifiers may be used to correlate and/or match primary data set <NUM> with a corresponding secondary data set on an IPFIX collector. Additionally or alternatively, primary data set <NUM> may include, identify, and/or be presented in a certain format. For example, primary data set <NUM> may include and/or identify a template (such as an option template) that defines the format of the corresponding data, common properties, and/or elements. In some embodiments, this template may be consistent with the definitions, formats, and/or configurations provided in the <NPL>,".

In some examples, secondary data set <NUM> may include and/or represent data derived, deduced, and/or extrapolated by the control path, control plane, and/or routing plane of a network device. In one example, secondary data set <NUM> may include, represent, and/or identify one or more common properties and/or elements of flows monitored across observation domains of an observation cloud. Examples of data, common properties, and/or elements incorporated in secondary data set <NUM> include, without limitation, egress interface indexes, gateway addresses, Simple Network Management Protocol (SNMP) indexes, interface names, ingress Virtual Routing and Forwarding (VRF) indexes, VRF names, next hop addresses, egress VRF indexes, portions of primary data set <NUM>, ingress interface indexes, next hop indexes, combinations or variations of one or more of the same, and/or any other suitable data, common properties, and/or elements.

In some examples, secondary data set <NUM> may include, specify, and/or identify one or more common properties identifiers. In such examples, these common properties identifiers may be used to correlate and/or match secondary data set <NUM> with primary data set <NUM> on an IPFIX collector. Additionally or alternatively, secondary data set <NUM> may include, identify, and/or be presented in a certain format. For example, secondary data set <NUM> may include and/or identify a template (such as an option template) that defines the format of the corresponding data, common properties, and/or elements. In some embodiments, this template may be consistent with the definitions, formats, and/or configurations provided in the IETF's RFC <NUM>.

As illustrated in <FIG>, example system <NUM> additionally includes a plurality of databases, such as database <NUM> and database <NUM>. Databases <NUM> and <NUM> are located on and/or maintained by an IPFIX collector. Database <NUM> is dedicated to store primary data sets observed by observation domains. Database <NUM> is dedicated to store secondary data sets derived by observation clouds.

As illustrated in <FIG>, example system <NUM> may also include one or more observation points, such as observation points <NUM>. In some examples, observation points <NUM> may include and/or represent physical devices and/or components that facilitate the flow of traffic within a network. Additionally or alternatively, observation points <NUM> may be implemented by physical devices and/or components that forward traffic within a network. Examples of such physical devices and/or components include, without limitation, physical interfaces, Gigabit Ethernet (GE) interfaces, <NUM>-Gigabit Ethernet (XE) interfaces, Ten GE interfaces, Asynchronous Transfer Mode (ATM) interfaces, Frame Relay interfaces, egress interfaces, ingress interfaces, communication ports, portions of one or more of the same, combinations or variations of one or more of the same, and/or any other suitable physical devices and/or components.

As illustrated in <FIG>, example system <NUM> may further include one or more observation domains, such as observation domain <NUM>. In some examples, observation domain <NUM> may include and/or represent a set and/or group of observation points. Additionally or alternatively, observation domain <NUM> may be implemented by physical devices and/or components that forward traffic within a network. In one example, each observation domain may include and/or be assigned a unique observation domain identifier. Examples of such physical devices and/or components include, without limitation, Packet Forwarding Engines (PFEs), Physical Interface Cards (PICs), Flexible PIC Concentrators (FPCs), Switch Interface Boards (SIBs), control boards, connector interface panels, line cards, portions of one or more of the same, combinations or variations of one or more of the same, and/or any other suitable physical devices and/or components.

In some embodiments, these observation points and observation domains may be consistent with the definitions, implementations, and/or configurations provided in the <NPL>,".

In addition, as illustrated in <FIG>, example system <NUM> may include one or more observation clouds, such as observation cloud <NUM>. In some examples, observation cloud <NUM> may include and/or represent a set and/or group of observation domains. Additionally or alternatively, observation cloud <NUM> may be implemented by physical devices and/or components that route traffic within a network. In one example, each observation cloud may include and/or be assigned a unique observation cloud identifier. Examples of such physical devices and/or components include, without limitation, routing engines, FPCs, routers (such as provider edge routers, hub routers, spoke routers, autonomous system boundary routers, and/or area border routers), switches, hubs, modems, bridges, repeaters, gateways (such as Broadband Network Gateways (BNGs)), portions of one or more of the same, combinations or variations of one or more of the same, and/or any other suitable physical devices and/or components.

An apparatus for offloading IPFIX lookup and translation operations from observation domains may include all or portions of example system <NUM>. In some examples, system <NUM> in <FIG> may be implemented in a variety of ways. For example, all or a portion of example system <NUM> may represent portions of example system <NUM> in <FIG>. As shown in <FIG>, system <NUM> may include a network <NUM> that facilitates communication among network device <NUM>, computing device <NUM>, computing device <NUM>, and/or computing device <NUM>.

As illustrated in <FIG>, network <NUM> may include and/or represent various network devices and/or nodes that form and/or establish communication paths and/or segments. For example, network <NUM> may include a network device <NUM> that forwards traffic between computing device <NUM> and computing device <NUM>. In one example, IPFIX exporters implemented on and/or by network device <NUM> may monitor traffic passing through network <NUM>. In this example, the IPFIX exporters may observe and/or derive common properties of such traffic across observation points and/or observation domains. These IPFIX exporters may export IPFIX data sets from network device <NUM> to a collector <NUM> implemented on computing device <NUM>.

In some examples, and as will be described in greater detail below, one or more of modules <NUM> may cause computing device <NUM> to (<NUM>) receive, by collector <NUM>, at least one IPFIX message from an IPFIX exporter (such as observation domains <NUM>(<NUM>)-(N) and/or observation cloud <NUM>) implemented on a network device <NUM>, (<NUM>) identify, within the IPFIX message, a data set exported by the IPFIX exporter implemented on the network device <NUM>, (<NUM>) identify, within the IPFIX message, a data-level indicator that indicates whether the data set is (A) a primary data set observed by one of observation domains <NUM>(<NUM>)-(N) or (B) a secondary data set derived by observation cloud <NUM>, (<NUM>) identify, by collector <NUM>, a database (such as database <NUM> or database <NUM>) that corresponds to the data-level indicator identified within the IPFIX message, (<NUM>) store the data set in the database in accordance with the data-level indicator, and then (<NUM>) perform at least one action based at least in part on the data set stored in the database.

In some examples, network device <NUM> and computing device <NUM>, <NUM>, and <NUM> may each generally represent any type or form of physical computing device capable of reading computer-executable instructions. Examples of network device <NUM> and computing device <NUM>, <NUM>, and <NUM> include, without limitation, routers (such as provider edge routers, hub routers, spoke routers, autonomous system boundary routers, and/or area border routers), switches, hubs, modems, bridges, repeaters, gateways (such as BNGs), multiplexers, network adapters, network interfaces, client devices, laptops, tablets, desktops, servers, cellular phones, Personal Digital Assistants (PDAs), multimedia players, embedded systems, wearable devices, gaming consoles, variations or combinations of one or more of the same, and/or any other suitable gateway devices.

Network <NUM> generally represents any medium or architecture capable of facilitating communication or data transfer. In one example, network <NUM> may include one or more of computing devices <NUM>, <NUM>, and <NUM> even though these devices are illustrated as being external to network <NUM> in <FIG>. Additionally or alternatively, network <NUM> may include other devices that facilitate communication among network device <NUM> and/or computing devices <NUM>, <NUM>, and <NUM>. Network <NUM> may facilitate communication or data transfer using wireless and/or wired connections. Examples of network <NUM> include, without limitation, an intranet, an access network, a layer <NUM> network, a layer <NUM> network, a Multiprotocol Label Switching (MPLS) network, an Internet Protocol (IP) network, a heterogeneous network (e.g., layer <NUM>, layer <NUM>, IP, and/or MPLS) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Personal Area Network (PAN), the Internet, Power Line Communications (PLC), a cellular network (e.g., a Global System for Mobile Communications (GSM) network), portions of one or more of the same, variations or combinations of one or more of the same, and/or any other suitable network.

<FIG> is a flow diagram of an example computer-implemented method <NUM> for offloading IPFIX lookup and translation operations from observation domains. The steps shown in <FIG> may be performed by any suitable computer-executable code and/or computing system, including system <NUM> in <FIG>, system <NUM> in <FIG>, system <NUM> in <FIG>, system <NUM> in <FIG>, and/or variations or combinations of one or more of the same. In one example, each of the steps shown in <FIG> may represent an algorithm whose structure includes and/or is represented by multiple sub-steps, examples of which will be provided in greater detail below.

As illustrated in <FIG>, at step <NUM> one or more of the systems described herein may receive, at an IPFIX collector, at least one IPFIX message from an IPFIX exporter implemented on a remote device. For example, receiving module <NUM> may, as part of computing device <NUM> in <FIG>, receive at least one IPFIX message from an IPFIX exporter implemented on network device <NUM>. In one example, the IPFIX exporter may include and/or represent one of observation domains <NUM>(<NUM>)-(N). In this example, observation domains <NUM>(<NUM>)-(N) may each be implemented and/or executed by an FPC and/or a line card installed to a routing engine of network device <NUM>.

In another example, the IPFIX exporter may include and/or represent observation cloud <NUM>. In this example, observation cloud <NUM> may be implemented and/or executed by a routing engine of network device <NUM>. Additionally or alternatively, observation cloud <NUM> may be implemented and/or executed by a physical processor of network device <NUM>.

The systems described herein may perform step <NUM> in a variety of ways and/or contexts. In some examples, receiving module <NUM> may monitor computing device <NUM> for IPFIX messages arriving from certain network devices within network <NUM>. In one example, the IPFIX exporter implemented on network device <NUM> may generate an IPFIX message for export to collector <NUM> implemented on computing device <NUM>. In this example, the IPFIX exporter may direct network device <NUM> to send the IPFIX message to computing device <NUM>. While monitoring computing device <NUM> for IPFIX messages, receiving module <NUM> may receive the IPFIX message as it arrives at collector <NUM> implemented on computing device <NUM>.

In one example, network device <NUM> in <FIG> may include and/or represent a router deployed within network <NUM>. In this example, observation cloud <NUM> in <FIG> may include and/or represent a routing engine of the router, and observation domains <NUM>(<NUM>)-(N) may each include and/or represent a line card installed to the routing engine of the router. Additionally or alternatively, observation points <NUM>(<NUM>) may include and/or represent a set of network interfaces incorporated in a first line card, and observation points <NUM>(N) may include and/or represent a set of network interfaces incorporated in a second line card.

Continuing with this example, identification module <NUM> may identify and/or detect various packets encountered at the network interfaces incorporated on the first line card. In this example, identification module <NUM> may search portions (e.g., headers, metadata, and/or payloads) of the packets for certain observable properties that those packets share in common. During the search, identification module <NUM> may identify and/or detect such observable properties in connection with the first line card. Examples of such observation properties include, without limitation, ingress interface indexes, next hop indexes, flow directions, forwarding statuses, data link frame sizes, data link frame selections, combinations or variations of one or more of the same, and/or any other suitable data, common properties, and/or elements.

In one example, identification module <NUM> may match and/or group the packets to one another at the first line card based at least in part on the observable properties that the packets share in common. This matching and/or grouping of packets may represent and/or constitute a flow across the entire first line card. In this example, identification module <NUM> may select and/or pick one of the observable properties to serve as a domain-level common properties identifier that is locally unique to the flow and/or the first line card.

In one example, exporter module <NUM> may generate and/or prepare an IPFIX message in connection with the first line card. In some embodiments, this IPFIX message may identify and/or specify the observable properties that the packets detected and/or observed at the first line card share in common. Additionally or alternatively, this IPFIX message may identify and/or include certain domain-level statistics of the packets detected and/or observed in connection with the first line card. In this example, exporter module <NUM> may determine such domain-level statistics that account for the packets detected and/or observed in connection with the first line card.

In some examples, exporter module <NUM> may insert the domain-level common properties identifier, the corresponding observable properties, and/or the domain-level statistics or information into the IPFIX message. In one example, the domain-level common properties identifier, the corresponding observable properties, and/or the domain-level statistics may collectively represent and/or constitute the data set of the IPFIX message.

Additionally or alternatively, exporter module <NUM> may set and/or configure a data-level indicator within the IPFIX message. This data-level indicator may indicate and/or specify whether the data set of the IPFIX message is a primary or secondary data set. Accordingly, because this IPFIX message originates from the first line card, the data-level indicator of this IPFIX message may be set and/or configured to indicate that the data set exported in the IPFIX message is a primary data set observed by the first line card. Exporter module <NUM> may then send this IPFIX message to collector <NUM>.

In some examples, the line cards may report to the routing engine certain information and/or statistics about various packets encountered by the respective network interfaces. In these examples, identification module <NUM> may receive, at the routing engine, notifications with such information and/or statistics from the line cards. For example, the notifications originating from the first line card may indicate that the network interfaces of that line card have detected certain packets that share observable properties in common. Similarly, the notifications originating from the second line card may indicate that the network interfaces of that line card have detected certain packets that share observable properties in common.

In some examples, identification module <NUM> may search the notifications from the line cards for indications of the observable properties that those packets share in common. During the search, identification module <NUM> may identify and/or detect such indications of the observable properties. The match and/or grouping of packets may represent and/or constitute a flow across the entire first line card. In this example, identification module <NUM> may maintain and/or copy the domain-level common properties identifiers from the notifications for use as cloud-level common properties identifiers.

In some examples, identification module <NUM> may derive, determine, and/or discover additional properties of the packets at the routing engine. In one example, identification module <NUM> may perform a lookup operation at the routing engine based at least in part on one or more of the observable properties of the packets. For example, identification module <NUM> may look up the OIF and/or gateway based at least in part on the destination prefix of the flow in question.

Additionally or alternatively, identification module <NUM> may perform a translation operation at the routing engine based at least in part on one or more of the observable properties of the packets. For example, identification module <NUM> may translate the looked up OIF and/or gateway based at least in part on the destination prefix of the flow in question. In this example, identification module <NUM> may be able to derive the additional properties of the packets at the routing engine based at least in part on the lookup operation and/or the translation operation.

In one example, identification module <NUM> may match and/or group the packets to one another at the routing engine based at least in part on the observable properties that the packets share in common. This matching and/or grouping of packets may represent and/or constitute a flow across the entire routing engine, which includes the first and second line cards. In this example, identification module <NUM> may maintain and/or copy the domain-level common properties identifiers from the notifications for use as cloud-level common properties identifiers.

In one example, exporter module <NUM> may generate and/or prepare an IPFIX message in connection with the routing engine. In some embodiments, this IPFIX message may identify and/or specify the derived properties that the packets detected and/or observed across all the line cards installed to the routing engine. Additionally or alternatively, this IPFIX message may identify and/or include certain cloud-level statistics of the packets detected and/or observed across all the line cards installed to the routing engine. In this example, identification module <NUM> may determine such cloud-level statistics that account for the packets detected and/or observed across all the line cards installed to the routing engine.

In some examples, exporter module <NUM> may insert the cloud-level common properties identifier, the corresponding derived properties, and/or the cloud-level statistics or information into the IPFIX message. In one example, the cloud-level common properties identifier, the corresponding derived properties, and/or the cloud-level statistics may collectively represent and/or constitute the data set of the IPFIX message. In this example, the cloud-level common properties identifier may enable collector <NUM> to record the data set as originating from the routing engine.

Additionally or alternatively, exporter module <NUM> may set and/or configure a data-level indicator within the IPFIX message. This data-level indicator may indicate and/or specify whether the data set of the IPFIX message is a primary or secondary data set. Accordingly, because this IPFIX message originates from the routing engine, the data-level indicator of this IPFIX message may be set and/or configured to indicate that the data set exported in the IPFIX message is a primary data set observed by the first line card. Exporter module <NUM> may then send this IPFIX message to collector <NUM>.

In some examples, the routing engine may export the common properties identifiers to the line cards. As a result, the line cards may be able to generate and/or export IPFIX messages to collector <NUM> without performing lookups or translations for the common properties (e.g., the OIF and gateway) of the destination prefix. In other words, the line cards may simply use these common properties identifiers, thereby avoiding lookup and translation.

<FIG> is a block diagram of an example system <NUM> for offloading IPFIX lookup and translation operations from observation domains. As illustrated in <FIG>, example system <NUM> may include and/or represent an exporter <NUM> and collector <NUM>. In some examples, exporter <NUM> may send an IPFIX message <NUM> to collector <NUM>. In one example, exporter <NUM> may be implemented and/or executed on network device <NUM> in <FIG>. Additionally or alternatively, collector <NUM> may be implemented and/or executed on computing device <NUM> in <FIG>.

In some examples, exporter <NUM> may include and/or represent one of observation domains <NUM>(<NUM>)-(N) implemented on network device <NUM> in <FIG>. In other examples, exporter <NUM> may include and/or represent observation cloud <NUM> implemented on network device <NUM> in <FIG>.

<FIG> is a block diagram of an example IPFIX message <NUM>. As illustrated in <FIG>, example IPFIX message <NUM> may include and/or contain a data-level indicator <NUM> and a data set <NUM>. In some examples, data-level indicator <NUM> may include and/or represent a bit of data that indicates to collector <NUM> whether data set <NUM> is a primary or secondary data set. Accordingly, data set <NUM> may include and/or represent a primary data set that originates from one of observation domains <NUM>(<NUM>)-(N). Alternatively, data set <NUM> may include and/or represent a secondary data set that originates from observation cloud <NUM>.

In some examples, IPFIX message <NUM> may be exported via a variety of different protocols. For example, if IPFIX message <NUM> carries a primary data set, IPFIX message <NUM> may be exported using a User Datagram Protocol (UDP) transport session. In this example, IPFIX message <NUM> may also include and/or contain a header with an observation domain identifier that uniquely corresponds to and/or identifies the observation domain from which IPFIX message <NUM> originated.

Alternatively, if IPFIX message <NUM> carries a secondary data set, IPFIX message <NUM> may be exported using a Transmission Control Protocol (TCP) transport session. In this example, IPFIX message <NUM> may also include and/or contain a header with an observation cloud identifier that uniquely corresponds to and/or identifies the observation cloud from which IPFIX message <NUM> originated.

<FIG> is an illustration of example primary data set <NUM>. As illustrated in <FIG>, example primary data set <NUM> may include and/or contain a data set and/or a format. In one example, the data set may include, represent, and/or identify an ingress interface index, a next hop index, a flow direction, a forwarding status, and/or a data link frame size. In this example, the data set may correspond and/or be specific to a particular flow detected and/or observed at one of observation domains <NUM>(<NUM>)-(N). More specifically, the data set may correspond and/or be specific to a particular flow detected and/or observed across various observation points within one of observation domains <NUM>(<NUM>)-(N).

In one example, the format of the data set may correspond to and/or be defined by a template. For example, the template may indicate that the format of the data set includes first and second common properties identifiers located in the left-most positions and first, second, and third common properties located in the right-most positions. In this example, exporter module <NUM> may select the ingress interface index and/or the next hop index to serve as the first and second common properties identifiers, respectively. These common properties identifiers may enable collector <NUM> to record primary data set <NUM> as originating from one of observation domains <NUM>(<NUM>)-(N). Additionally or alternatively, these common properties identifiers may enable collector <NUM> to correlate and/or match primary data set <NUM> to the corresponding secondary data set at a later point in time.

<FIG> is an illustration of example secondary data set <NUM>. As illustrated in <FIG>, example secondary data set <NUM> may include and/or contain a data set and/or a format. In one example, the data set may include, represent, and/or identify an ingress interface index, an ingress interface SNMP ID, an ingress interface name, ingress VRF ID, ingress VRF name, a next hop index, an egress interface SNMP ID, an egress interface name, a next hop IPv4 address, an egress VRF ID, and/or an egress VRF name. In this example, the data set may correspond and/or be specific to a particular flow detected and/or observed at observation cloud <NUM>. More specifically, the data set may correspond and/or be specific to a particular flow detected and/or observed across various observation domains <NUM>(<NUM>)-(N) included in observation cloud <NUM>. For example, primary data set <NUM> in <FIG> and secondary data set <NUM> in <FIG> may correspond to and/or represent the same flow.

In one example, the format of the data set may correspond to and/or be defined by a template. For example, the template may indicate that the format of the data set includes first and second common properties identifiers located in the left-most positions of a separate data rows. In this example, the template may also indicate that first, second, third, and fourth common properties follow the first common properties identifier in one of the data rows and fifth, sixth, seventh, eighth, and ninth common properties follow the second common properties identifier in another data row.

In one example, exporter module <NUM> may select the ingress interface index and/or the next hop index to serve as the first and second common properties identifiers. By doing so, exporter module <NUM> may ensure that the first and second common properties identifiers are the same in both primary data set <NUM> and secondary data set <NUM>. These common properties identifiers may enable collector <NUM> to record secondary data set <NUM> as originating from observation cloud <NUM>. Additionally or alternatively, these common properties identifiers may enable collector <NUM> to correlate and/or match secondary data set <NUM> to the corresponding primary data set at a later point in time.

In some examples, network device <NUM> may install, in a FIB at the routing engine of network device <NUM>, a set of routes capable of carrying and/or forwarding traffic within network <NUM>. Once the set of routes are installed, the set of routes may be considered a full and/or complete FIB. In one example, the routing engine of network device <NUM> may withhold some of the routes installed in the FIB from each of the line cards such that none of the line cards includes all the routes. By doing so, the routing engine may enable network device <NUM> to support the IPFIX protocol while also enabling the line cards to operate with less memory capacity than traditional IPFIX technologies.

Returning to <FIG>, at step <NUM> one or more of the systems described herein may identify, within the IPFIX message, a data set exported by the IPFIX exporter implemented on the remote device. For example, identification module <NUM> may, as part of computing device <NUM> in <FIG>, identify a data set <NUM> within IPFIX message <NUM>. In one example, data set <NUM> may be exported by one of observation domains <NUM>(<NUM>)-(N) implemented on network device <NUM>. In this example, data set <NUM> may be exported by observation cloud <NUM> implemented on network device <NUM>.

The systems described herein may perform step <NUM> in a variety of ways and/or contexts. In some examples, identification module <NUM> may search IPFIX message <NUM> for any data sets exported from an IPFIX exporter. During the search, identification module <NUM> may identify data set <NUM> as being exported by an IPFIX exporter implemented on network device <NUM>. In one example, data set <NUM> may include and/or represent a primary data set that identifies observable properties of a particular flow encountered across one of observation domains <NUM>(<NUM>)-(N). In another example, data set <NUM> may include and/or represent a second data set that identifies derived properties of a particular flow encountered across observation domains <NUM>(<NUM>)-(N) of observation cloud <NUM>.

Returning to <FIG>, at step <NUM> one or more of the systems described herein may identify, within the IPFIX message, a data-level indicator that indicates whether the data set is a primary data set observed by an observation domain or a secondary data set derived by an observation cloud. For example, identification module <NUM> may, as part of computing device <NUM> in <FIG>, identify data-level indicator <NUM> that indicates whether data set <NUM> is a primary data set observed by an observation domain or a secondary data set derived by an observation cloud. In one example, data-level indicator <NUM> may indicate that data set <NUM> was exported by an observation domain. In another example, data-level indicator <NUM> may indicate that data set <NUM> was exported by an observation cloud.

The systems described herein may perform step <NUM> in a variety of ways and/or contexts. In some examples, identification module <NUM> may search IPFIX message <NUM> for any indication as to whether data set <NUM> is a primary or secondary data set. For example, identification module <NUM> may search the header and/or metadata of IPFIX message <NUM>. During this search, identification module <NUM> may identify data-level indicator <NUM>. Afterward, collector module <NUM> may be able to determine whether data set <NUM> is a primary or secondary data set.

Returning to <FIG>, at step <NUM> one or more of the systems described herein may identify, at the IPFIX collector, a database that corresponds to the data-level indicator identified within the IPFIX message. For example, identification module <NUM> may, as part of computing device <NUM> in <FIG>, identify database <NUM> as corresponding to data-level indicator <NUM> identified within IPFIX message <NUM>. In this example, data-level indicator <NUM> may indicate that IPFIX message <NUM> is carrying a primary data set.

Alternatively, identification module <NUM> may identify database <NUM> as corresponding to data-level indicator <NUM> identified within IPFIX message <NUM>. In this example, data-level indicator <NUM> may indicate that IPFIX message <NUM> is carrying a secondary data set.

Returning to <FIG>, at step <NUM> one or more of the systems described herein may store the data set in the database in accordance with the data-level indicator. For example, storage module <NUM> may, as part of computing device <NUM> in <FIG>, store data set <NUM> in database <NUM> in accordance with data-level indicator <NUM>. In one example, if data-level indicator <NUM> indicates that data set <NUM> is a primary data set, storage module <NUM> may store data set <NUM> in database <NUM>. In another example, if data-level indicator <NUM> indicates that data set <NUM> is a secondary data set, storage module <NUM> may store data set <NUM> in database <NUM>.

The systems described herein may perform step <NUM> in a variety of ways and/or contexts. In some examples, storage module <NUM> may copy data set <NUM> from IPFIX message <NUM> to database <NUM> or database <NUM>. Additionally or alternatively, storage module <NUM> may maintain and/or preserve data set <NUM> in database <NUM> or database <NUM> to facilitate certain actions and/or services based at least in part on the unification of IPFIX exports.

In some examples, modules <NUM> may repeat any of the steps and/or actions described above in connection with additional IPFIX messages received by collector <NUM>. For example, one or more of modules <NUM> may cause computing device <NUM> to (<NUM>) receive, by collector <NUM>, at least one additional IPFIX message from an IPFIX exporter (such as observation domains <NUM>(<NUM>)-(N) and/or observation cloud <NUM>) implemented on network device <NUM>, (<NUM>) identify, within the additional IPFIX message, a data set exported by the IPFIX exporter implemented on the network device <NUM>, (<NUM>) identify, within the additional IPFIX message, an additional data-level indicator that indicates whether the data set is (A) a primary data set observed by one of observation domains <NUM>(<NUM>)-(N) or (B) a secondary data set derived by observation cloud <NUM>, (<NUM>) identify, by collector <NUM>, a database (such as database <NUM> or database <NUM>) that corresponds to the data-level indicator identified within the IPFIX message, and then (<NUM>) store the data set in the database in accordance with the data-level indicator.

In some examples, collector <NUM> may include, provide, and/or deploy a union layer, such as union module <NUM>, that performs and/or executes a union of database <NUM> and database <NUM>. In one example, the union performed by union module <NUM> may involve identifying one or more common properties identifiers included in primary data set <NUM>. In this example, the union may also involve searching database <NUM> for secondary data set <NUM> based at least in part on the common properties identifier(s).

Additionally or alternatively, the union performed by union module <NUM> may involve identifying one or more common properties identifiers included in secondary data set <NUM>. In this example, the union may also involve searching database <NUM> for primary data set <NUM> based at least in part on the common properties identifier(s). In addition, the union may involve creating a record that unifies and/or joins primary data set <NUM> and secondary data set <NUM> based at least in part on the common properties identifier.

As a specific example, union module <NUM> may identify the ingress interface index and the next hop index as the first and second common properties identifier of a particular flow. In this example, union module <NUM> may search database <NUM> and/or database <NUM> for any entries that include and/or identify the same ingress interface index and the next hop index. By doing so, union module <NUM> may be able to correlate and/or match one entry from database <NUM> and another entry from database <NUM> based at least in part on the ingress interface index and the next hop index.

Continuing with this example, union module <NUM> may create a record <NUM> in <FIG>. As illustrated in <FIG>, record <NUM> may include and/or represent a unified data set formed from the union of primary data set <NUM> and secondary data set <NUM>. For example, the unified data set may include, represent, and/or identify the ingress interface index SNMP ID, the ingress interface name, the ingress VRF name, the egress interface SNMP ID, the egress interface name, the flow direction, the forwarding status, and/or the data link size frame of the flow in question.

<FIG> is flow diagram of an example computer-implemented method <NUM> for storing and/or unifying primary and secondary data sets at an IPFIX collector. The steps shown in <FIG> may be performed by any suitable computer-executable code and/or computing system, including system <NUM> in <FIG>, system <NUM> in <FIG>, system <NUM> in <FIG>, system <NUM> in <FIG>, and/or variations or combinations of one or more of the same. In one example, each of the steps shown in <FIG> may represent an algorithm whose structure includes and/or is represented by multiple sub-steps.

As illustrated in <FIG>, collector <NUM> may receive an IPFIX message and then determine whether the IPFIX message includes an information element in the data set. If the IPFIX message does include an information element in the data set, collector <NUM> may determine whether the data set is a primary or secondary data set. If it is a secondary data set, collector <NUM> may determine whether the data set originated from an observation domain or an observation cloud. On the one hand, if the data set originated from an observation domain, collector <NUM> may store the information element in an observation domain database and then access the next information element in the dataset. On the other hand, if the data set originated from an observation cloud, collector <NUM> may store the information element in an observation cloud database and then access the next information element in the dataset.

If it is a primary data set, collector <NUM> may determine whether the data set originated from an observation domain or an observation cloud. On the one hand, if the data set originated from an observation domain, collector <NUM> may retrieve the common properties identifier from the observation domain database. On the other hand, if the data set originated from an observation cloud, collector <NUM> may retrieve the common properties identifier from the observation cloud database.

If the common properties identifier is found, collector <NUM> may expand at least one of the data sets stored in the databases (e.g., the primary data set stored in the observation domain database) to include and/or represent the union and/or combination of the primary data set and the secondary data set. Accordingly, the resulting entry and/or record in the database may be complete and/or whole. If, however, the common properties identifier is not found, collector <NUM> may increment the counter and/or drop the primary data set.

If the data set did not originate from either an observation domain or an observation cloud, collector <NUM> may perform and/or execute normal IPFIX processing on the information element and then access the next information element in the data set. Afterward, collector <NUM> may repeat the foregoing steps for each information element included in the data set until completion of the processing of the entire data set.

Returning to <FIG>, at step <NUM> one or more of the systems described herein may perform at least one action based at least in part on the data set stored in the database. For example, action module <NUM> may, as part of computing device <NUM> in <FIG>, perform at least one action based at least in part on the data set stored in database <NUM> and/or database <NUM>. Examples of such actions include, without limitation, traffic metering, traffic profiling, security evaluations, intrusion detection, accounting services, billing services, combinations or variations of one or more of the same, and/or any other suitable actions.

As explained above in connection with <FIG>, the various systems and methods described herein may be able to offload IPFIX lookup and translation operations from observation domains to observation clouds. ASIC lookups and translations are very costly for IPFIX exporters. For any derived fields of an IP prefix, an IPFIX exporter may need to perform a lookup and a translation. However, for any observed fields of an IPFIX prefix, an IPFIX exporter may now be able to avoid performing a lookup and a translation. Accordingly, to facilitate IPFIX exports without forcing ASIC lookups and translation on observation domains, the duties of exporting data to an IPFIX collector may be divided among the observation domains and an observation cloud that includes a complete FIB table. In this example, the observation cloud may be able to perform lookups and translations based at least in part on the destination prefix instead of the observation domains. As a result, the observation domains need not download the complete FIB table, nor do the observation domains need to have the memory capacity to store the complete FIB table.

In one example, the source address and destination address may match across <NUM>,<NUM> flows encountered and/or observed at a router. In this example, a routing engine of the router may export the common properties identifier as a primary data set. Additionally or alternatively, the OIF and gateway may be common across all line cards installed to the routing engine. In this example, the routing engine of the router may export those matching common properties. That way, each line card installed to the routing engine may not need to export those matching common properties individually. Accordingly, the routing engine of the router may be able to export common properties to avoid downloading the complete FIB table to each line card.

In one example, the routing engine may be able to export the common properties identifiers to the line cards so that, when the line cards generate or export IPFIX packets to the collector, the line cards need not perform lookups or translations for the OIF and gateway of the destination prefix. In this example, the line cards may simply use these common properties identifiers, thereby avoiding lookup and translation. This division of IPFIX export duties may enable certain modern ASIC chipsets incorporated into line cards to still contribute and/or support the IPFIX protocol.

In some examples, each line card installed to the routing engine of the router may export IPFIX messages. In such examples, these IPFIX messages may each include a header with a field for the corresponding observation domain identifier. This observation domain identifier may uniquely identify and/or specify the exporting line card.

Similarly, the routing engine of the router may export IPFIX messages. In such examples, these IPFIX messages may each include a header with a field for the corresponding observation cloud identifier. This observation cloud identifier may uniquely identify and/or specify the exporting routing engine.

The collector maintains two databases, one for primary data sets and another one for secondary data sets. The primary data sets arrives at the collector via IPFIX messages exported by observation domains (e.g., line cards and/or PFE instances running on the line cards). In contrast, the secondary data sets arrives at the collector via IPFIX messages exported by observation clouds (e.g., routing engines and/or routers). The collector may perform lookups based on the common properties identifiers to combine and/or unify the primary and secondary data sets. The collector may then create a unified and/or combined record with the primary and secondary data sets.

<FIG> is a block diagram of an example computing system <NUM> capable of implementing and/or being used in connection with one or more of the embodiments described and/or illustrated herein. In some embodiments, all or a portion of computing system <NUM> may perform and/or be a means for performing, either alone or in combination with other elements, one or more of the steps described in connection with <FIG>. All or a portion of computing system <NUM> may also perform and/or be a means for performing and/or implementing any other steps, methods, or processes described and/or illustrated herein.

Computing system <NUM> broadly represents any type or form of electrical load, including a single or multi-processor computing device or system capable of executing computer-readable instructions. Examples of computing system <NUM> include, without limitation, workstations, laptops, client-side terminals, servers, distributed computing systems, mobile devices, network switches, network routers (e.g., backbone routers, edge routers, core routers, mobile service routers, broadband routers, etc.), network appliances (e.g., network security appliances, network control appliances, network timing appliances, SSL VPN (Secure Sockets Layer Virtual Private Network) appliances, etc.), network controllers, gateways (e.g., service gateways, mobile packet gateways, multi-access gateways, security gateways, etc.), and/or any other type or form of computing system or device.

Computing system <NUM> may be programmed, configured, and/or otherwise designed to comply with one or more networking protocols. According to certain embodiments, computing system <NUM> may be designed to work with protocols of one or more layers of the Open Systems Interconnection (OSI) reference model, such as a physical layer protocol, a link layer protocol, a network layer protocol, a transport layer protocol, a session layer protocol, a presentation layer protocol, and/or an application layer protocol. For example, computing system <NUM> may include a network device configured according to a Universal Serial Bus (USB) protocol, an Institute of Electrical and Electronics Engineers (IEEE) <NUM> protocol, an Ethernet protocol, a T1 protocol, a Synchronous Optical Networking (SONET) protocol, a Synchronous Digital Hierarchy (SDH) protocol, an Integrated Services Digital Network (ISDN) protocol, an Asynchronous Transfer Mode (ATM) protocol, a Point-to-Point Protocol (PPP), a Point-to-Point Protocol over Ethernet (PPPoE), a Point-to-Point Protocol over ATM (PPPoA), a Bluetooth protocol, an IEEE <NUM>. XX protocol, a frame relay protocol, a token ring protocol, a spanning tree protocol, and/or any other suitable protocol.

Computing system <NUM> may include various network and/or computing components. For example, computing system <NUM> may include at least one processor <NUM> and a system memory <NUM>. Processor <NUM> generally represents any type or form of processing unit capable of processing data or interpreting and executing instructions. For example, processor <NUM> may represent an ASIC, a system on a chip (e.g., a network processor), a hardware accelerator, a general purpose processor, and/or any other suitable processing element.

Processor <NUM> may process data according to one or more of the networking protocols discussed above. For example, processor <NUM> may execute or implement a portion of a protocol stack, may process packets, may perform memory operations (e.g., queuing packets for later processing), may execute end-user applications, and/or may perform any other processing tasks.

System memory <NUM> generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and/or other computer-readable instructions. Examples of system memory <NUM> include, without limitation, Random Access Memory (RAM), Read Only Memory (ROM), flash memory, or any other suitable memory device. Although not required, in certain embodiments computing system <NUM> may include both a volatile memory unit (such as, for example, system memory <NUM>) and a non-volatile storage device (such as, for example, primary storage device <NUM>, as described in detail below). System memory <NUM> may be implemented as shared memory and/or distributed memory in a network device. Furthermore, system memory <NUM> may store packets and/or other information used in networking operations.

In certain embodiments, example computing system <NUM> may also include one or more components or elements in addition to processor <NUM> and system memory <NUM>. For example, as illustrated in <FIG>, computing system <NUM> may include a memory controller <NUM>, an Input/Output (I/O) controller <NUM>, and a communication interface <NUM>, each of which may be interconnected via communication infrastructure <NUM>. Communication infrastructure <NUM> generally represents any type or form of infrastructure capable of facilitating communication between one or more components of a computing device. Examples of communication infrastructure <NUM> include, without limitation, a communication bus (such as a Serial ATA (SATA), an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI), a PCI Express (PCIe), and/or any other suitable bus), and a network.

Memory controller <NUM> generally represents any type or form of device capable of handling memory or data or controlling communication between one or more components of computing system <NUM>. For example, in certain embodiments memory controller <NUM> may control communication between processor <NUM>, system memory <NUM>, and I/O controller <NUM> via communication infrastructure <NUM>. In some embodiments, memory controller <NUM> may include a Direct Memory Access (DMA) unit that may transfer data (e.g., packets) to or from a link adapter.

I/O controller <NUM> generally represents any type or form of device or module capable of coordinating and/or controlling the input and output functions of a computing device. For example, in certain embodiments I/O controller <NUM> may control or facilitate transfer of data between one or more elements of computing system <NUM>, such as processor <NUM>, system memory <NUM>, communication interface <NUM>, and storage interface <NUM>.

Communication interface <NUM> broadly represents any type or form of communication device or adapter capable of facilitating communication between example computing system <NUM> and one or more additional devices. For example, in certain embodiments communication interface <NUM> may facilitate communication between computing system <NUM> and a private or public network including additional computing systems. Examples of communication interface <NUM> include, without limitation, a link adapter, a wired network interface (such as a network interface card), a wireless network interface (such as a wireless network interface card), and any other suitable interface. In at least one embodiment, communication interface <NUM> may provide a direct connection to a remote server via a direct link to a network, such as the Internet. Communication interface <NUM> may also indirectly provide such a connection through, for example, a local area network (such as an Ethernet network), a personal area network, a wide area network, a private network (e.g., a virtual private network), a telephone or cable network, a cellular telephone connection, a satellite data connection, or any other suitable connection.

In certain embodiments, communication interface <NUM> may also represent a host adapter configured to facilitate communication between computing system <NUM> and one or more additional network or storage devices via an external bus or communications channel. Examples of host adapters include, without limitation, Small Computer System Interface (SCSI) host adapters, Universal Serial Bus (USB) host adapters, IEEE <NUM> host adapters, Advanced Technology Attachment (ATA), Parallel ATA (PATA), Serial ATA (SATA), and External SATA (eSATA) host adapters, Fibre Channel interface adapters, Ethernet adapters, or the like. Communication interface <NUM> may also enable computing system <NUM> to engage in distributed or remote computing. For example, communication interface <NUM> may receive instructions from a remote device or send instructions to a remote device for execution.

As illustrated in <FIG>, example computing system <NUM> may also include a primary storage device <NUM> and/or a backup storage device <NUM> coupled to communication infrastructure <NUM> via a storage interface <NUM>. Storage devices <NUM> and <NUM> generally represent any type or form of storage device or medium capable of storing data and/or other computer-readable instructions. For example, storage devices <NUM> and <NUM> may represent a magnetic disk drive (e.g., a so-called hard drive), a solid state drive, a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash drive, or the like. Storage interface <NUM> generally represents any type or form of interface or device for transferring data between storage devices <NUM> and <NUM> and other components of computing system <NUM>.

In certain embodiments, storage devices <NUM> and <NUM> may be configured to read from and/or write to a removable storage unit configured to store computer software, data, or other computer-readable information. Examples of suitable removable storage units include, without limitation, a floppy disk, a magnetic tape, an optical disk, a flash memory device, or the like. Storage devices <NUM> and <NUM> may also include other similar structures or devices for allowing computer software, data, or other computer-readable instructions to be loaded into computing system <NUM>. For example, storage devices <NUM> and <NUM> may be configured to read and write software, data, or other computer-readable information. Storage devices <NUM> and <NUM> may be a part of computing system <NUM> or may be separate devices accessed through other interface systems.

Many other devices or subsystems may be connected to computing system <NUM>. Conversely, all of the components and devices illustrated in <FIG> need not be present to practice the embodiments described and/or illustrated herein. The devices and subsystems referenced above may also be interconnected in different ways from those shown in <FIG>. Computing system <NUM> may also employ any number of software, firmware, and/or hardware configurations. For example, one or more of the example embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, software applications, computer-readable instructions, or computer control logic) on a computer-readable medium. The term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, without limitation, transmission-type media, such as carrier waves, and non-transitory-type media, such as magnetic-storage media (e.g., hard disk drives and floppy disks), optical-storage media (e.g., Compact Disks (CDs) and Digital Video Disks (DVDs)), electronic-storage media (e.g., solid-state drives and flash media), and other distribution systems. A computer readable medium may include non-transitory type media such as physical storage media including storage discs and solid state devices. A computer readable medium may also or alternatively include transient media such as carrier signals and transmission media. A computer-readable storage medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

While the foregoing disclosure sets forth various embodiments using specific block diagrams, flowcharts, and examples, each block diagram component, flowchart step, operation, and/or component described and/or illustrated herein may be implemented, individually and/or collectively, using a wide range of hardware, software, or firmware (or any combination thereof) configurations. In addition, any disclosure of components contained within other components should be considered as examples in nature since many other architectures can be implemented to achieve the same functionality.

In some examples, all or a portion of system <NUM> in <FIG> may represent portions of a cloud-computing or network-based environment. Cloud-computing and network-based environments may provide various services and applications via the Internet. These cloud-computing and network-based services (e.g., software as a service, platform as a service, infrastructure as a service, etc.) may be accessible through a web browser or other remote interface. Various functions described herein may also provide network switching capabilities, gateway access capabilities, network security functions, content caching and delivery services for a network, network control services, and/or and other networking functionality.

In addition, one or more of the modules described herein may transform data, physical devices, and/or representations of physical devices from one form to another. Additionally or alternatively, one or more of the modules recited herein may transform a processor, volatile memory, non-volatile memory, and/or any other portion of a physical computing device from one form to another by executing on the computing device, storing data on the computing device, and/or otherwise interacting with the computing device.

The process parameters and sequence of the steps described and/or illustrated herein are given by way of example only and can be varied as desired.

The preceding description has been provided to enable others skilled in the art to best utilize various aspects of the example embodiments disclosed herein. This example description is not intended to be exhaustive or to be limited to any precise form disclosed. Many modifications and variations are possible without departing from the scope of the instant disclosure. The embodiments disclosed herein should be considered in all respects illustrative and not restrictive. Reference should be made to the appended claims in determining the scope of the instant disclosure.

Claim 1:
A method comprising:
receiving (<NUM>), at an Internet Protocol Flow Information Export "IPFIX" collector, at least one IPFIX message from an IPFIX exporter implemented on a remote device, the remote device comprising a router;
identifying (<NUM>), within the IPFIX message, a data set exported by the IPFIX exporter implemented on the remote device;
identifying (<NUM>), within the IPFIX message, a data-level indicator that indicates whether the data set is:
a primary data set observed by an observation domain implemented on one or more line cards installed to a routing engine of the remote device; or
a secondary data set derived by an observation cloud implemented on the routing engine of the remote device,
wherein the derived data is obtained through lookup and/or translation at the routing engine and
the observed data is obtained without lookup or translation at the routing engine;
identifying (<NUM>), at the IPFIX collector, a database of a plurality of databases that corresponds to the data-level indicator identified within the IPFIX message, wherein the plurality of databases comprises a first database (<NUM>) dedicated to storing primary data sets and a second database (<NUM>) dedicated to storing secondary data sets;
storing (<NUM>) the data set in the database identified in accordance with the data-level indicator; and
performing (<NUM>) at least one action based at least in part on the data set stored in the identified database.