Patent Description:
Internet networking technology has revolutionised our lives in recent decades. An internet network provider provides its users with ability to access content from various sources, and the content downloaded by the users typically includes audio-data, video-data, online-messaging, website-browsing, social-media browsing and file transfer (e.g., including the use of Facebook™, Instagram™, and Whatsapp™) and so on.

It is desirable for the network providers to understand how their network is being used, and what type of content is being accessed by the users. Large data streams, mainly video-data, constitute a majority of network traffic today. Currently, network providers have very limited visibility into the traffic travelling over their network, and this limited visibility hinders the network provider's ability to identify and resolve data capacity problems faced by the network. Currently, they are addressing data capacity problems by increasing the bandwidth of their networks, which is an expensive solution.

In order to better manage data traffic (for quality and cost reasons), it would be advantageous for the network providers to have visibility into microscopic aspects, such as how many video streams are concurrently active at a time, what their durations are, what resolutions they operate at, and how often they adapt their rate. Visibility into these attributes can allow them to better understand both content characteristics and data viewing patterns, so they can implement useful changes to tune their network to meet content-provider expectations and enhance user experience.

There are two major technologies currently being used to understand network traffic. The first technology is hardware-based and is known as Deep Packet Inspection (DPI). This technology analyses each and every packet travelling through the network using hardware that is very expensive, both economically and computationally (because a high processing power is required to analyse each and every packet). Another disadvantage of this technique is that it provides limited scalability. For at least these reasons, DPI is not practical to implement for most network operators.

The second technology makes use of packet inspection software for packet-analyses and flow-analyses separately. This technique is also computationally very expensive and is of limited scalability.

Video traffic is rapidly increasing every day and it is supposed to increase even further in the near future as higher resolutions (e.g., 1440p and <NUM>) become more prevalent, and augmented and virtual reality begin to take off.

For at least the above reasons, network providers need better visibility into their networks, in particular to solve network capacity problems in an efficient and cost effective manner, and to improve user experience.

Subject matter relevant for the present invention is described in <CIT>, and in published document <NPL>.

It is desired, therefore, to provide a network traffic monitoring process and system that overcome or alleviate one or more difficulties of the prior art, or to at least provide a useful alternative.

In accordance with one embodiment of the present invention, there is provided a network traffic monitoring process executed by a network traffic monitoring system of a communications network, according to claim <NUM>. In some embodiments, the flow types include video flows and non-video flows.

In some embodiments, the flow types include video flows of respective different resolutions.

In some embodiments, the process includes determining service providers of at least some of the large network flows from DNS information.

In some embodiments, the flow metrics include metrics of burstiness at respective time scales.

In some embodiments, the time scales represent a geometric series.

In accordance with one embodiment of the present invention, there is provided a network traffic monitoring process executed by a software-defined networking (SDN) flow switch of a communications network, according to claim <NUM>.

In accordance with some embodiments of the present invention, there is provided a network traffic monitoring system according to claim <NUM>.

In some embodiments, the system includes a user interface component configured to receive user requests and, responsive to the requests, to generate user interface data representing an interactive user interface for displaying information on large network flows detected by the system, the information including classifications of the large network flows.

In accordance with some embodiments of the present invention, there is provided at least one computer-readable storage medium having stored thereon executable instructions, according to claim <NUM>.

It is known that conventional method of deep packet inspection monitors every single packet of each data stream. Considering the size of network traffic, this is unscalable, takes a long time, and is extremely expensive to implement.

Also described herein, monitoring of network traffic is achieved by combining packet-level monitoring with flow-level monitoring. In an embodiment, the step of monitoring data of a data stream comprises the step of obtaining data packets until a threshold is reached. Advantageously, in an embodiment, therefore only some of the data packets are monitored (e.g. the first few Mega-Bytes of every data stream). In an embodiment, this limited packet inspection provides sufficient information to determine data type, content provider's information, address of content request, and so on. In an embodiment, this is followed by flow-level monitoring of the data stream, which can be used to implement a classification analysis to classify the data streams of the identified data type into different data categories.

In an embodiment, the threshold is chosen to trigger flow level monitoring for data types which comprise large volume data flows. These are otherwise known as "elephants", and include large downloads, video streaming, augmented reality, virtual reality data streams and other large data flows. Data flows that do not achieve the threshold generally comprise small data flows ("mice"), such as social network posts and the like. While mice comprise the majority of data types, elephants take up most volume of data traffic. Ignoring the mice, means that it is possible for embodiments of the invention to concentrate on the elephants, being the large data volume flows.

The method described herein is highly scalable because only a limited number of data packets of a data stream undergo packet-level monitoring. This provides a low cost and highly scalable solution for monitoring and classifying network traffic. Further, the method concentrates on the large volume data flows and ignores the mice, further optimising processing.

The step of obtaining the data packets may comprises mirroring data packets of the data stream.

It is an advantage of at least an embodiment of the present invention that the data packets of the data stream being examined are not affected or modified. This is because the packet inspection is performed on mirrored data packets.

As described herein, the step of obtaining the data packets of a data stream is stopped when the threshold is reached and the data type of the data stream is determined.

As described herein, the step of monitoring data is implemented via a software defined networking (SDN) solution. As described herein, flow telemetry is implemented by utilising hardware counters.

The balance between hardware and software processing reduces costs, increases scaleability, and enables extraction of enough information from the data for implementation of a classification analysis.

As described herein, the method comprises the further step of carrying out a classification analysis to classify the predetermined data type into one of a plurality of data categories. The data categories may comprise classifying into data resolution e.g. high definition, medium definition, low definition. The categories may also comprise data relating to a provider identity (e.g. Netflix™, YouTube™ etc.). The categorisation may comprise identifying type of data e.g. video, large download etc..

As described herein, the classification of the predetermined data type is based on characteristics of the data, comprising one or more of: scanned profile, size of data stream, resolution and data provider's information.

As described herein, a process of machine learning is implemented to improve the classification analysis.

Also described herein is an apparatus for monitoring data traffic over a network, the data traffic comprising a plurality of data streams, the apparatus comprising at least one processor arranged to monitor data in each data stream and determine data type for each data stream, and arranged to implement flow telemetry for a predetermined at least one of the data types, to determine flow volume for each data stream of the predetermined data type.

The processor may comprise a software defined network application which is arranged to instruct obtaining data packets of the data stream until a threshold is reached. In an embodiment, the processor comprises a large flow detector arranged to inspect the data packets. In an embodiment the data packets are obtained by mirroring data packets of the data stream.

The processor is arranged to collect hardware counters to implement the flow telemetry.

As described herein, the processor is arranged to examine the predetermined data type and classify the data streams of the data type into one of a plurality of data categories. In an embodiment, machine learning is used to profile the data streams and categorise them. A database is provided to store characteristics of data streams to enable classification.

In an embodiment, the apparatus comprises a user interface presenting information on the data types and categories and flow stream analysis.

Also described herein is a computer program, arranged to instruct a processor to implement any of the above methods.

Also described herein is a non-volatile computer readable medium, providing a computer program in accordance with the third aspect of the invention.

Also described herein is a data signal comprising any of the above computer programs.

Some embodiments of the present invention are hereinafter described, by way of example only, with reference to the accompanying drawings, in which:.

Embodiments of the present invention include a network traffic monitoring system and process that are able to classify data packets flowing through a communications network into different network flows, and to characterise those flows by type and traffic properties. Although some embodiments of the present invention are described below in the context of monitoring flows of video data in a communications network, it should be understood that the network traffic monitoring apparatus and process are not limited to video data but can be generally applied to identify and characterising flows of any type of network traffic in a communications network.

Software Defined Networking (SDN) is a flexible and versatile networking technology which uses a centralized control system that is separated from network switches and other network devices. The centralized SDN control system uses an SDN control protocol such as OpenFlow to configure SDN network devices such as network switches. In conventional networking, each switch has its own independent control software for deciding where to move data packets. However, in an SDN system, the decisions of packet-movement are ultimately made by the centralized SDN controller which controls the behaviour of the SDN switches to process packets accordingly. The SDN controller can be custom programmed, based on the network operator's needs and independent of the individual switches.

An SDN switch generally includes flow tables that define matching rules to identify whether a network packet received at an input port of the switch belongs to any of a plurality of defined or predetermined flows (also known in the art as 'packet flows', 'network flows', and 'traffic flows'), and for each such flow, an action to perform on packets belonging to the flow, typically identifying a corresponding exit port of the switch to which packets of that flow are to be output from the switch. As indicated above, the flow tables of an SDN switch can be dynamically modified by an SDN controller via an SDN control protocol such as the Open Flow protocol.

The inventors have determined that an SDN-based system is well suited for identifying and classifying network traffic flows (including video traffic flows) traversing through a communications network. The inventors have developed an SDN-based apparatus that includes an independently programmable controller and SDN switches, which in the described embodiments are low cost off-the-shelf OpenFlow switches. This system operates at a much higher speed in comparison to conventional DPI and packet inspection software processes.

<FIG>shows the architecture and functional blocks of a network traffic monitoring apparatus applied to a carrier network, in accordance with an embodiment of the present invention. In this embodiment, the network traffic monitoring apparatus can be transparently inserted between two ports of a network where network traffic monitoring (video monitoring in the described embodiment) is desired. The apparatus <NUM> is inserted between an internet gateway <NUM> and an access gateway <NUM> of the network. The end user (on the very left of <FIG>) can be connected to the network through the access gateway <NUM> using either wired (DSL, Ethernet, Fiber) and/or wireless (e.g. <NUM>/<NUM>, WiFi) technology. The video content providers are on the right, connected to the carrier/enterprise network through the Internet gateway <NUM>.

The apparatus <NUM> can be inserted into any desired link as a 'bump-in-the-wire' where network data inspection is required.

As shown in <FIG>, the apparatus <NUM> includes an SDN switch <NUM>, a large flow detector <NUM>, a data broker <NUM>, a user interface <NUM>, a Database <NUM>, and an SDN Application <NUM> on an SDN controller <NUM>.

Network traffic from the content provider enters the apparatus <NUM> from the internet gateway <NUM>, and exits at the access gateway <NUM> and towards the end user. Typically, the network traffic includes all sorts of data flows, including streamed video files, streamed audio files, large download files, small data flows representing social-media browsing and mobile application messaging, and so on.

In the described embodiment, the video files streamed by users through the network are monitored as follows.

In an example greenfield installation, the SDN switch <NUM> is initially configured to mirror all of the data packets of every incoming flow to the large flow detector <NUM>. The large flow detector <NUM> keeps track of the volume of each flow until a pre-determined threshold flow volume is reached or exceeded, and then it notifies the data broker. In one embodiment, the pre-determined threshold volume is in the range of <NUM> to <NUM> Mega-bytes, depending upon the type of video flows to be identified. In another embodiment, the threshold volume is set to <NUM> Mega-Bytes. If the flow volume is greater than the corresponding threshold, then it is deemed to be a "heavy-flow" (or as an "Elephant-flow", using a term of art). The heavy-flow can either be a video stream or a large-sized downloadable file or downloadable video whose flow volume and duration are larger than the pre-determined threshold volume and period. Once an elephant-flow is identified, the data broker <NUM> instructs the SDN application <NUM> to insert a reactive flow-entry for this specific flow into the SDN switch <NUM>, and to stop the mirroring of packets for this flow. This relieves the large flow detector <NUM> from performing further analysis of the elephant-flow. As a result, the scalability of the large flow detector <NUM> is substantially improved in comparison to conventional DPI and software-inspection systems.

Once an elephant-flow has been identified and a reactive entry for the elephant flow is saved in a flow-table of the SDN switch <NUM>, the data broker <NUM> polls the counters of the SDN switch <NUM> periodically to develop a traffic profile for this elephant-flow. In this specification, a traffic profile of a flow includes information regarding the identity of the flow and the identity of the content provider of that flow. <FIG> respectively represent internal modules of the data broker <NUM> and the SDN Application <NUM> that collect telemetry, develop traffic profiles, and perform the flow identification and classification processes. The data broker <NUM> includes two intelligent processes, namely: (i) a video-identifier, and (ii) a video-classifier. Different types of elephant-flows have different traffic rate profiles. Based on these characteristics, the video-identifier is used to identify video streams from the other types of traffic flows of the identified elephant-flows. Further, the video-classifier is used to classify the identified video streams by their resolutions.

The SDN switch <NUM> communicates with the SDN controller <NUM> using an OpenFlow protocol. The SDN switch <NUM> acts as a hardware filter that limits the fraction of traffic (typically to the first few Mega-Bytes of traffic from a flow) mirrored for flow analysis, while the SDN application <NUM> creates reactive flow-table entries for elephant flows that are then monitored via the hardware counters and (Group) Table <NUM>. The thresholds are tuned on flow volume and duration at which a reactive flow-entry gets created, and the inventors have found empirically that a value of <NUM> Mega-Bytes for volume threshold works well - this keeps the hardware flow-mod operations to less than <NUM>% of all flows (in the inventors' trials over <NUM>% of flows are short), while limiting the packet mirroring to the large flow detector <NUM> to less than one-third of link traffic (since around <NUM>% of traffic volume is carried in elephant flows). This balance between hardware and software processing reduces cost, increases scalability, and enables extraction of enough information for machine learning algorithms to achieve high classification accuracy.

<FIG> illustrates a multiple flow-table structure of the SDN switch <NUM>. These flow tables of the SDN switch <NUM> are configured to identify and categorise incoming flows. Table <NUM> and Table <NUM> are a reactive flow table and a proactive flow table, respectively, and are used to store reactive and proactive entries, respectively. Table <NUM> is a default flow table, and table <NUM> is a group table. Using the flow tables, a match command is used to identify known incoming flows, and corresponding action commands are used to perform an appropriate action of moving the flow to the corresponding entry in the group table (Table <NUM>).

Reactive rules of Table <NUM> match on <NUM>-tuples for known flows. A <NUM>-tuple is an ordered set of five values that identify a flow. Reactive rules of Table <NUM> are of highest priority, and are installed as a consequence of elephant flows identified by the large flow detector <NUM>. They automatically time out (and are removed from the table) upon a pre-defined period of inactivity ranging from <NUM> seconds to <NUM> seconds. The reactive flow entries achieve two objectives: (i) to stop mirroring elephant-flow packets to the software large flow detector <NUM>, and (ii) to provide flow-level telemetry (flow characteristics) for the individual (potentially video) elephant-flows. The action corresponding to a match in the reactive table (Table <NUM>) sends the flow to its appropriate entry in the group table (Table <NUM>), which identifies the content provider (YouTube, Netflix etc.). The content provider for the flow is identified by searching for the server IP address in the most recent captured DNS suffixes (e.g. googlevideo. com or nflxvideo. com) that are stored in a time-series database table (the "flow DB" in <FIG>) by the large flow detector <NUM>. If a video stream from a new DNS suffix is detected (e.g. ttnvw. net), then a new group entry (for Twitch in this example) is created dynamically in the group table. This not only makes the apparatus <NUM> adaptive to new video content providers, but also allows tracking aggregate video volumes for each video content provider by storing them in the group table. Therefore, the reactive flow table is used for fine grain visibility whereas the group table is used for coarse level visibility of video flows detected by the apparatus <NUM>.

Proactive entries (Table <NUM>) are statically pushed by the SDN controller <NUM> so that all Transmission Control Protocol (TCP) (proto=<NUM>) and User Datagram Protocol (UDP) (proto=<NUM>) packets received from the content provider, that have not already matched an elephant flow (Table <NUM>), are forwarded to port-<NUM> (i.e. access gateway <NUM>) and mirrored at port-<NUM> by the SDN switch <NUM> to the large flow detector <NUM>. This includes DNS reply packets that contain the domain names of video content providers and the video server IP addresses. All other types of packets are sent to Table <NUM>, where the default action is to cross-connect the input (internet gateway <NUM>) and output (access Gateway <NUM>) ports without performing any mirroring or processing.

The apparatus <NUM> does not send any data packets to the SDN controller <NUM>, thereby minimizing the load on the SDN controller <NUM>, reducing packet-forwarding latency, and immunizing against failures of the SDN controller <NUM>.

It is an advantage of the apparatus <NUM> that it is completely transparent to the network. This is because the SDN switch <NUM> makes copies of the packets that require monitoring and sends them to the large flow detector <NUM>. The SDN switch <NUM> forwards one copy of the data packets to their traffic path without interruption. The apparatus <NUM> does not modify packets.

Another advantage of the apparatus <NUM> is that it does not overload the SDN controller <NUM>. The SDN switch <NUM> does not send any data packets to the SDN controller <NUM>; instead, any packets that need to be inspected are sent as copies to the large flow detector <NUM>. This protects the SDN controller <NUM> from overload from the data-plane, allowing it to service other SDN applications.

The large flow packet detector <NUM> is also responsible for keeping track of new flows, including <NUM>-tuple information, duration, and volume, using efficient in-memory data structures. If a flow is active for more than a threshold volume, it is deemed to be an elephant flow, and the large flow detector <NUM> informs the Broker <NUM>, which then makes a RESTful API call to the SDN controller <NUM> to insert a corresponding reactive flow-table entry into the SDN switch <NUM>. This suppresses data-plane traffic for this flow from being mirrored to the large flow detector <NUM>, and also triggers telemetry for that elephant flow.

The other responsibility of the large flow detector <NUM> is detection of DNS A-type replies, upon which it extracts the domain name and server IP addresses, and sends these via JSON to the data broker <NUM>, which writes it into a time-series DNS database table of the database <NUM>. This database <NUM> is used to associate each video stream with its content provider.

The data broker <NUM> queries per-flow statistics (counters), stores them in a time-series flow database table ("Flow DB" in <FIG>) with timestamp information representing a corresponding timestamp of each query (e.g., the current time), and exposes the stored data to the user interface via appropriate RESTful APIs. The telemetry collects per-flow (fine grain) and per-group (coarse grain) usage statistics using the Stats collector module of our SDN application.

<NUM> Video Identification: In accordance with the above discussion, the large flow detector <NUM> identifies all elephant flows, which may include a mixture of video streams and other elephant flows, and then stops their packets from being mirrored.

A video identification process is executed to distinguish video streams from elephant transfers, and to identify their content providers and resolutions. At a high level, the video identification process: (a) determines attributes of a given flow, which are then fed into an intelligent classifier to distinguish video streams from elephant transfers, (b) queries the DNS database ("DNS DB" in <FIG>) using the flow's client/server IP address to associate the video stream with its content provider, and (c) estimates the resolution of the video stream (in the described embodiment, the resolution being estimated as one of Low, Medium, High, or Ultra-high).

<NUM> Usage Collection and Storage: The data broker <NUM> collects counter data representing flow counters per content provider (group table) and per video stream reactive flow table entry. While the number of entries in the group table is generally relatively small and fixed, the number of reactive flow entries can vary significantly with time. Polling the latter when the number of entries is large can result in a multipart reply - for example a Noviflow SDN switch <NUM> breaks the response into chunks of <NUM> flows each - putting considerable strain on the agent in the switch <NUM>, and consequently affecting timeliness of the results. To mitigate this effect, in the described embodiment the apparatus <NUM> tunes the polling frequency depending on the number of entries in the reactive flow table. Specifically, when the number of reactive flow entries is less than <NUM>, the apparatus <NUM> polls the counters every second, but reduces the polling frequency to once every <NUM> seconds when the number of entries exceeds <NUM>,<NUM>. When the data broker <NUM> stores counter data received from the SDN switch <NUM>, it stores the received counter data together with corresponding timestamp information so that flow profiles representing the temporal characteristics of each flow can be generated. The flow/group-level counters are thus stored in a time-series Flow DB, as shown in <FIG>, and are periodically sent in a JSON-formatted message to a machine learning process of the data broker <NUM> as described below.

The data broker <NUM> executes a machine learning classification processes to determine whether traffic pertaining to a flow is streaming video or not (a "video identifier" process), and if so, to determine the video stream resolution (a "resolution classifier" process).

<NUM> Attributes: Attribute selection is of paramount importance for training classifiers, given that classifiers should be predictive to correctly identify and classify video streams. <FIG> shows plots of traffic patterns observed for various video streams of different content providers, for example, Youtube™, Netflix™ and Twitch™ (at different resolutions: low, medium, high and ultra-high definition), and other elephant flows including those of the Facebook™ application and large downloads (representative of bulk transfers or GoogleDrive™ or Dropbox™ cloud storage synchronization) during the first three minutes of their activity.

It can be seen that, due to the buffering that accompanies video streaming, the idle-time characteristic (i.e., the fraction of time that no data is exchanged) of video flows in <FIG> is quite distinctive compared to the large download flow in <FIG>). The average rate (shown by dotted red lines) of the Youtube™ 2160p (<NUM> ultrahigh definition video) in <FIG> is much higher than that of other video resolutions (shown in <FIG> and <FIG>), but is comparable to the large download in <FIG>. In addition to idle-time and average rate, the burstiness characteristic of each flow is also distinctive - the low resolution video and the large download exhibit the most and the least bursty patterns respectively, among these representative profiles shown in <FIG>. Based on these visual observations, it is evident that idle-time, average rate and burstiness are collectively able to identify and classify video flows. For example, the Facebook™ application flow shown in <FIG> exhibits similar characteristics of video streams (shown in <FIG> ) in terms of idle-time and burstiness, but its rate is far below those of video streams.

The average rate and fraction of idle-time for a flow can be computed over a moving window (of say one minute). Burstiness of flow traffic can be computed in various ways, and it is noted (particularly in the characterisation of long-range dependent traffic) that it should be measured at multiple time-scales. Accordingly, in the described embodiments a coefficient of variance (i.e. the ratio of the standard deviation to the mean, CV = σ/µ) is computed for streams at time-granularities of <NUM>, <NUM>, <NUM>, <NUM> and <NUM> seconds to provide respective values denoted herein as σ<NUM>/µ, σ<NUM>/µ, σ<NUM>/µ, σ<NUM>/µ, and σ<NUM>/µ. These burstiness measures, in addition to the idle-time and average rate µ of each flow, are provided as attributes to the classifiers. Note that, for a new flow, there may be only a subset of burstiness attributes at the beginning, because computing σ<NUM> would require collection of data for at least a minute. A flow that commenced only <NUM> seconds ago would only be able to yield σ<NUM>/µ, σ<NUM>/µ and σ<NUM>/µ since there are fewer than <NUM> data points at time scales of <NUM>-seconds and <NUM>-seconds.

As described above, the data broker <NUM> executes two classifiers, namely the video identifier (to indicate whether a flow is a streaming video or not), and the resolution classifier (to determine the resolution of a video stream during playback). Each classifier is invoked periodically (every <NUM> seconds in the described embodiment) - initial invocation may have access to only five attributes (idle-time, µ, σ<NUM>/µ, σ<NUM>/µ, and σ<NUM>/µ), and subsequent invocations that have access to more (burstiness related) attributes may change the classification, improving accuracy and/or identifying resolution changes. The training of the classifiers is described below.

An embodiment of the apparatus was built using open source software components is shown in block diagram form in <FIG>. This apparatus <NUM> identifies and classifies video streams in real-time at line-rates up to <NUM> Gbps. In this embodiment, the SDN application is implemented on top of the open source Ryu SDN controller (as described at https://osrg. io/ryu/), augmented by the open source Bro packet inspection engine (https://www. org/) for flow state management and event triggering, and the databases are generated using the InfluxDB time-series database platform (https://www. influxdata. com/), open source relational database PostgreSQL (https://www. postgresql. org/), and CouchDB (http://couchdb. org/), and a web-GUI written using the ReactJS Javascript GUI library (https://reactjs. org/) for user interaction. Further, each of these components runs in a separate docker container or virtual machine (VM) in a cloud environment provided by the VMware Esxi <NUM> hypervisor. Each of the VMs runs the Ubuntu server <NUM> LTS operating system, and is allocated a four-core CPU with <NUM> GB of memory and <NUM> GB of disk space.

This apparatus <NUM> is currently managing three environments: (a) an SDN-enabled experimental university campus network spanning several WiFi access points, (b) a point-to-point link over which an industrial scale Spirent traffic generator feeds traffic into our setup, and (c) a live campus dormitory network link operating at <NUM> Gbps and serving several hundred real users.

<NUM> SDN switch: The SDN switch <NUM> is a fully Openflow <NUM> compliant NoviSwitch <NUM>, as shown in <FIG>. It provides <NUM> Gbps of throughput, tens of thousands of TCAM flow entries, and millions of exact-match flow-entries in DRAM.

<NUM> Large flow detector: The Bro (v2. <NUM>) open-source tool <NUM> is used for inspection of the mirror traffic. The event-handlers were written in Bro to keep track of flow duration and volume, and to trigger an API call to the data broker when an elephant flow is detected. Similarly, DNS replies are also parsed and the information passed to the data broker <NUM> for recording into the time-series database.

<NUM> Data Broker: The data broker <NUM> in this embodiment is written in the Python language. The data broker <NUM> receives the <NUM>-tuple of elephant flows and DNS information from the Bro large flow detector <NUM>, inserts/modifies flow/group entries, and collects statistical data from the SDN application <NUM> via a RESTful API. Flow and group statistics collected from the SDN application <NUM> are written into a time series InfluxDB database <NUM>. Flow level information is queried from the InfluxDB database <NUM> periodically for processing by the intelligent classifier powered by the Weka tool (v3. <NUM>) (as described at https://en. org/wiki/Weka_(machine_learning)) using Weka's Python wrapper interface (v0. The intelligent classifier identifies video flows, queries the DNS database to label video flows, calls RESTful APIs to modify flow entries' output group, and identifies video stream resolutions.

<NUM> SDN controller and application: A Ryu (v4. <NUM>) Openflow controller <NUM> is used in this embodiment. The SDN application <NUM> is written in Python and exposes northbound RESTful APIs to the data broker <NUM> for inserting or modifying network rules and polling flow statistics. Successful RESTful API calls result in appropriate actions (e.g., network rule insertion, modification and counter collection) at the SDN switch <NUM> serving the data-plane.

<NUM> Data Bases: There are three databases in the system <NUM> to store flow usage statistics, DNS information, and system configurations. The time-series InfluxDB (v1. <NUM>) <NUM> is used to store periodic flow/group statistics. In the same InfluxDB <NUM>, information of DNS A-type replies is also stored, including the domain name and client/server IP addresses. An object relational database PostgreSQL (v9. <NUM>) is used to store the mapping between domain IP addresses and domain name suffix. A NoSQL CouchDB (v2. <NUM>) document-oriented database is used to store configurations of the SDN switch <NUM> such as Open Flow DataPath ID (DPID) and multi-table configurations.

<NUM> Web Interface: The apparatus <NUM> provides an interactive graphical user interface (GUI) or 'front-end' <NUM> for network administrators to visualize video streams in their network, implemented in ReactJS using the Rubix template and the D3 library. Example screenshots are shown in <FIG> and <FIG>.

The classifiers of the apparatus <NUM> were trained with datasets collected by the apparatus <NUM> itself. In order to have the ground truth for the training, a Python script was written to generate video streaming from various providers, namely Youtube™, Netflix™, Youku™, Facebook™, Tencent™, and other long duration traffic, including large downloads (e.g., Google-Drive sync) and dynamic webpages (i.e., Office <NUM>, Facebook homepage, WhatsApp), over an experimental WiFi SDN network called "uniwide_sdn". The Youtube Player API was used to stream videos at specified resolutions, namely low:144p, 240p, 360p; medium: 480p, 720p; high: 1080p, 1440p; and ultra-high: <NUM>.

For the purpose of training, the scripts limit each flow (video and non-video) to <NUM> seconds (i.e. about two minutes), even though every chosen video had a total length in excess of <NUM> minutes. Internet browser Firefox™ version <NUM> was used to play the videos. The scripts played videos from the top <NUM> most popular providers, at different video resolutions, as well as different large ISO files for download and Google-Drive sync, so as to diversify the training datasets.

At the end of each two-minute activity, the script queried the InfluxDB <NUM> to extract the flow profile (byte counts at <NUM>-second time interval) and calculate the attributes as described above. The <NUM>-second traffic profile was then split into <NUM> sub-profiles (corresponding to time intervals of [<NUM>,<NUM>]s, [<NUM>,<NUM>]s, [<NUM>,<NUM>]s, [<NUM>,<NUM>]s, [<NUM>, <NUM>]s, [<NUM>, <NUM>]s, [<NUM>, <NUM>]s, and [<NUM>, <NUM>]s). The script lastly computed the attributes for each of the sub profiles. Note that the short sub-profiles (e.g. [<NUM>,<NUM>]s) will have incomplete attributes such as σ<NUM>/µ and σ<NUM>/µ. The script was run for <NUM> weeks, collecting a total of <NUM>,<NUM> labelled training instances for elephant flows (video and non-video), of which <NUM>,<NUM> instances were labelled by video resolution.

<FIG> shows the resulting histograms of each attribute for the video identifier, and the differences are visually apparent. For example, the idle-time histogram in <FIG> shows that the idle-times of non-video flows are centered at about <NUM>% with minor deviations, whereas the idle-times of video traffic flows are widely spread between <NUM>% and <NUM>%. The video and non-video streams are not very distinct in their histogram of average rate in <FIG>. However, they are quite different in their burstiness behaviour at various time-scales, as seen in <FIG>.

<FIG> shows the attribute distributions for the resolution classifier. As expected, as the resolution increases from low to ultra-high, the average rate distribution shifts to the right (<FIG>), while the idle-time fraction distribution shifts to the left (<FIG>). The burstiness at various time-scales also decreases, as shown in <FIG>.

<NUM> Cross Validation: The Weka tool was used to train and validate the machine learning method for video identification and classification. Three popular classification algorithms were employed, namely J48, Random Forest, and MLP, that use the attributes described above. The efficacy of the classifiers was validated using the <NUM>-fold cross-validation method.

The cross-validation method randomly splits the dataset into training (<NUM>% of total instances) and validation (<NUM>% of total instances) sets. This cross-validation is repeated <NUM> times. The results are then averaged to produce a single performance metric. The accuracy of the video identifier is shown in the form of a confusion matrix in <FIG>. Over <NUM>% of video streams are correctly identified using the J48 and MLP algorithms, while the random forest has a slightly worse performance. The correct identification of non-video flows is over <NUM>% with J48, though Random forest and MLP perform worse. Overall, the J48 gives reasonable performance, with false positives (non-video being classified as video) below <NUM>% and false negatives (video being classified as non-video) below <NUM>%.

Confusion matrices for the resolution classifier are shown in <FIG>. Both J48 and Random forest yield a consistent overall accuracy of over <NUM>%. It is seen that high definition videos are wrongly classified more often than other resolutions, and are more likely to be mis-classified as medium resolution. Unsurprisingly, mis-classified low resolution videos are also more likely to be labelled as medium resolution. The geometry of the training instances is more suitable for decision-tree-based classifiers (i.e. J48 and Random forest) than neural-network-based classifiers (i.e. MLP), resulting in better accuracy. Furthermore, all of the chosen attributes have significant contributions in identifying/ classifying video traffic. and since J48 uses one decision tree for all training instances, it outperforms Random forest which employs a collection of independent decision trees, each considering a random subset of training instances.

Weka was used to evaluate the average merit of each attribute in the classification process. <FIG> shows that the idle-time and the burstiness at <NUM>-second and <NUM>-second (σ2/µ and σ4/µ) are the most important attributes to identify a video stream (shown by blow bars). However, average rate (µ) and idle-time contribute more to the resolution classifier.

The accuracy of machine learning was evaluated using a combination of instances from various sub-profiles (from the first <NUM> seconds to past one minute over a two-minute lifespan). The performance of the classifiers for each sub-profile was studied separately. <FIG> suggests that video streams are identified with an accuracy of about <NUM>% if only the first <NUM> seconds of their profile is available to the classifier. It is seen that the growth in the length of sub-profiles enhances the accuracy significantly - after <NUM> seconds, <NUM>% accuracy is achieved. Similarly, the accuracy of the resolution classifier is highly correlated with the length of sub-profile, as shown in <FIG>. This is not surprising, as various attributes computed during the first <NUM> seconds do not perfectly identify/classify video flows due to their initial buffering. For example, an ultra-high resolution video (<FIG>) is very similar to a large download if the idle-time, average rate and burstiness are considered for only the initial <NUM> or <NUM> seconds of the profile. The attributes σ8/µ and σ16/µ become available respectively only after <NUM> and <NUM> seconds of stream activity, and are fairly important for the classification.

<NUM> Summary: Identifying video streams and their resolutions for elephant flows based on their flow-level (rather than packet-level) characteristics such as idle-time, average-rate, and burstiness at multiple time-scales is feasible in real-time. <FIG> confirms that the apparatus <NUM> can correctly identify video flows with about <NUM>% accuracy within the first <NUM> seconds, rising to over <NUM>% accuracy in two minutes. Similarly, resolution classification achieves over <NUM>% accuracy in <NUM> seconds, rising to over <NUM>% in two minutes.

In this section, the efficacy of the system is disclosed by stressing it with a large number of emulated flows using a Telescope shows (by purple line) an average load around <NUM> Mbps within a second, which is very close to the rate of <NUM>. 56Mbps reported by the Spirent statistics (i.e. an error of less than <NUM>%). It is noted that the throughput of mirrored traffic (shown by yellow line) peaks at <NUM> Mbps and falls to zero gradually in <NUM> seconds.

This is not surprising, because the approach adopted in the present system only needs the initial few seconds worth of traffic from each new video flow to be sent to the traffic analyser for inspection; thereafter, a reactive flow entry is inserted to stop the packet mirroring. The mirror load is directly impacted by the rate of arrival of new video streams. Upon insertion of the reactive flow, no packet from that stream is mirrored, and our application thereafter polls byte-counts to monitor stream activity.

The Spirent statistics revealed that <NUM> GB of data were transferred, corroborating closely with the <NUM> GB measured by our system application. Of this, <NUM> GB was mirrored to the large flow detector <NUM>, corresponding to about <NUM>% of overall traffic. <FIG> shows the detection of elephant flows by our system, and corresponding reactive flow entries are pushed at the rate of <NUM> flows-per-second, resulting in almost zero packets being sent to the software large flow detector about <NUM> minutes into the experiment. The stress-test was meant to validate our system scalability to large number of active flows (<NUM>) and high rate of new flows (<NUM>/second), ensuring that both the software large flow detector <NUM> and the Openflow switch <NUM> can keep up. The deployments described next were found to have much lower requirements in terms of active flow numbers and new flow arrivals, even though the absolute data rates were higher.

The apparatus <NUM> was also tested for several months in the university dorm wired network serving hundreds of students.

The following discussion provides insights regarding video viewing patterns in the dorm, pertaining to the month from <NUM> May <NUM> to <NUM> May <NUM>. <FIG> shows a pie chart of the fraction of streams from the most popular video content providers - it is not unexpected that free video content providers (Youtube and Facebook) are the most dominant, at <NUM>% and <NUM>% respectively. Interestingly, the number of video streams from the gamer platform Twitch (<NUM>%) exceeds the number of Netflix streams (<NUM>%). It is noted that <NUM>% of video flows are sourced from Akamai media servers (i.e. akamai. net and akamaiedge. Lastly, the system allowed identification of many other cloud video providers such as Tencent, Youku, Amazon, Fastly, Alibaba, Shifen - these are grouped as "Others" in <FIG> that collectively contribute to <NUM>% of video streams in the dorm.

<FIG> depicts the complementary cumulative distribution function (CCDF) of the duration and average-rate of video streams from <NUM> popular content providers including Facebook™, Youtube™, Twitch™ and Netflix™, during May <NUM>. As shown in <FIG>, Twitch and Netflix videos are played for longer durations (with an average duration of about <NUM> minutes), followed by Youtube and Facebook videos with average durations of about <NUM> and <NUM> minutes respectively in the dorm. Considering the average-rate in <FIG>, Twitch and Netflix videos normally consume more bandwidth than Youtube and Facebook videos - Twitch and Netflix use on average <NUM> Mbps, while this measure is <NUM> and <NUM> Mbps for Youtube and Facebook, respectively.

In <FIG> the day-by-day video consumption pattern over the month is shown. Interesting observations that emerge from this are that there is a substantial fluctuation in the relative proportion of video providers from day to day, and it would seem that the dorm residents tended to watch Twitch gaming videos more on weekends than on weekdays. <FIG> shows the fraction of video streams at different resolutions on an hourly basis (averaged over the month of May <NUM>). Surprisingly, a majority of videos are playing at medium resolution and only a small fraction of videos are at ultra-high resolution, despite the university campus network having abundant bandwidth and rarely experiencing congestion. This is because most free movies (or long video clips) are only available at medium resolution or less (i.e. 144p, 240p, 360p, 480p and 720p) on Youtube and Facebook.

Nevertheless, the number of video streams by hour, along with the distribution of their quality, gives visibility into video streaming in the University dorm network that was not feasible before, and is much appreciated by the university IT staff who can obtain weekly and monthly reports directly from the apparatus <NUM>.

The described embodiments of the present invention judiciously combine software packet-level inspection with hardware flow-level telemetry, together with machine learning, to identify and classify video flows in real-time and at low-cost.

The above embodiments and examples have been described in the context of applications for identifying and classifying video data flowing through a network. However, it should be understood that the invention is not limited to monitoring video data and can be used to monitor other types of network data.

Claim 1:
A network traffic monitoring process executed by a network traffic monitoring system (<NUM>) of a communications network, the process including:
receiving mirrored data packets from a software-defined networking, SDN, flow switch (<NUM>) of a communications network;
processing header fields of the received data packets to identify subsets of the data packets as belonging to respective network flows;
detecting large network flows by determining, for each of the network flows, a corresponding cumulative amount of data contained in the received packets for the network flow until the cumulative amount of data reaches or exceeds a predetermined threshold amount of data;
for each detected large network flow, sending flow identification data to the SDN flow switch (<NUM>) to allow the SDN flow switch (<NUM>) to identify further packets of the large network flow as being packets of the large network flow and to stop mirroring the further packets of the large network flow to a component of the network traffic monitoring system;
for each large network flow, periodically receiving from the SDN flow switch (<NUM>) corresponding counter data representing a corresponding cumulative amount of data contained in packets of the large flow;
for each large network flow, processing the corresponding counter data and corresponding timestamp data to generate temporal metrics of the large network flow; and
for each large network flow, processing the generated temporal metrics with a trained classifier to classify the large network flow as being one of a plurality of predetermined flow types, and wherein
the flow metrics include idle time, average rate, and metrics of burstiness.