Flow Neural Network for Traffic Flow Modelling in IP Networks

1 INTRODUCTION . Deep Learning ( DL ) has gained substantial popularity in light of its applicability to real-world tasks across computer vision , natural language processing ( Goodfellow et al. , 2016 ) , protein structure prediction ( Senior et al. , 2020 ) and challenging games such as Go ( Silver et al. , 2017 ) . Typically , the data for these learning tasks takes the form of either grids , sequences , graphs or their combinations . The tremendous efforts on customizing neural network structures ( Krizhevsky et al. , 2012 ; Kiros et al. , 2015 ; Hochreiter & Schmidhuber , 1997 ) and learning strategies ( Sermanet et al. , 2018 ; Oord et al. , 2019 ) to explore the data-specific properties underpin the success of modern DL in these domains . Following the same design philosophy , we wish to capitalize on these advancements to develop a customized neural network and self-supervised learning strategy to tackle the crucial and timely challenge of traffic flow modelling in IP networks . 1.1 TRAFFIC FLOW MODELLING IN IP NETWORKS . An IP network is a communication network that uses Internet Protocol ( IP ) to send and receive messages between one or more devices such as computers , mobile phones . The messages could be general application data such as video , emails or control signals of any connected devices . When sending the messages from a source to a destination , the source device encapsulates the bit chunks of encoded messages into a set of IP packets . The packets then travel through communications links and routers or switches in a given routing path sequentially , thus forming the traffic flows in an IP network ( Hunt , 1992 ) . As one of the most commonly used global networks , the IP network provides the majority of such data transmission services to support today ’ s Internet applications such as video streaming , voice-over-IP , and Internet of Things . Therefore , a good understanding of the behaviorial patterns of the underlying traffic flows plays a crucial role in network planning , traffic management , as well as optimizing Quality of Service ( QoS , e.g. , transmission rate , delay ) . This challenge is termed as traffic flow modelling and is fundamental to IP networking research and practice . However , the high nonlinearity , randomness and complicated self similarity ( Leland et al. , 1994 ) of these traffic thwart extensive traditional analytical and learning models , particularly at fine-grained time scales , such as traffic flow modelling at a sub-second level . Consider the illustrative example in Fig . 1 , which depicts multiple packet flows with shared forwarding nodes and links in their routing paths . The sender of each flows streams data packets to the receiver at a dynamic sending rate , which is determined according to many factors such as its rate demand , existing traffic loads , available link bandwidth , and etc . The packets usually experience various delays on the journey due to actions such as forwarding processing , link transmission , packet queueing . For example , when the sum rate of Sender 2 and 3 exceeds 10 Gbps , the router R2–R4 will hold off and cache the arriving packets in their buffers until the links from R2 to Receiver 1 become free , causing what is known as the queueing delay . The extent of these delays depends on multiple factors , including the amount of traffic going on , the capacity of the router ’ s output queue , link bandwidth etc . The random establishment , interaction and termination of massive flow connections give rise to network dynamics . This illustrates the complexity of traffic flow modelling in IP network even for the simple example . This challenge is exacerbated when the traffic loads are running at over 100 Gbps and in a network with significantly larger size in practice . 1.2 MOTIVATING FLOWNN BASED TRAFFIC FLOW MODELLING . A flow pattern can be defined as anything that follows a trend and exhibits some kind of regularity , e.g. , distribution , periodicity etc . The modelling of traffic flow patterns can be done mathematically or by the use of data-driven learning algorithms . We argue that developing a customized FlowNN in the context of IP traffic flow modelling is important in two aspects : 1 ) improving the performances of supported network applications from the accurate modelling towards the behavioral patterns of traffic flows in IP network , particularly at the time scale of sub-second level ; 2 ) providing an exciting new “ playground ” and neural network model for the DL community to solve real-world-motivated research challenges by deeply combining its structure and working mechanisms . Next , we make the following two clarifications . Why not using traditional mathematical models . The past decades have seen numerous traffic models proposed to mathematically model the traffic characteristics of networks ( Gebali , 2015 ) . For example , extensive studies use the Poisson model to characterize the traffic by assuming the arrival pattern between two successive packets follows Poisson process . Considering the heavy tailed distribution and burstiness of the data-center traffic , recent work in Benson et al . ( 2010 ) models the traffic arrival pattern as a log-normal process . To capture the temporal patterns and make predictions accordingly , Seasonal Autoregressive Integrated Moving Average ( SARIMA ) is exploited in ( Ergenc & Ertan , 2019 ) to model the traffic time series . These analytical models may generate outputs that are easier to interpret , but are bonded to the specific working circumstance and assumptions . More importantly , these statistical models function at coarse time scales of hours and assume relatively smoother traffic patterns . However , as reported in many practical traffic measurements in e.g . Benson et al . ( 2010 ; 2011 ) ; Greenberg et al . ( 2009 ) , most flows last less than 1 minute . This implicates tasks requiring traffic models at finer-grained time scales are beyond the capability of these traditional models . Fig . 2 plots the traffic traces we collected from a practical backbone network–WIDE1 , which shows the realistic traffic patterns when the packet flows are sampled by two different time scales . The long time-scale plot in Fig . 2b shows clear a “ tide-effect ” associated with daily human activities . By contrast , the traffic traces in Fig . 2a get more noisy and difficult to recognize obvious patterns when they are counted by millisecond . 1http : //mawi.wide.ad.jp/˜agurim/index.html Why not using existing neural network models . When put in the context of data-driven learning , the traffic flow modelling problem can be reduced to the representation learning task . If treating the traffic flows as the general spatio-temporal data , extensive existing neural networks fit such task , including Convolutional Neural Net ( CNN , ( Mozo et al. , 2018 ) ) , Graph Neural Net ( GNN , ( Rusek et al. , 2019 ) ) , Recurrent Neural Net ( RNN ) as well as their variants and combinations ( e.g. , STHGCN ( Kalander et al. , 2020 ) , STGCN ( Yu et al. , 2018 ) , and Xiao et al . ( 2018 ) ; Polson & Sokolov ( 2017 ) ; Cui et al . ( 2020 ) ; Guo et al . ( 2019 ) ; Lin et al . ( 2019 ) ) . The customized designs to the data-specific properties make the success of these existing models , such as the convolutional operation to capture the spatially local correlations in CNN and the aggregation operation to extract the adjacent link correlations in GNN , and so on . As a human-engineered industrial system with clear system structure and working mechanism , the IP network creates domain-specific spatio-temporal data correlations , which are difficult to capture for the incumbent spatio-temporal models if without any modification . One of the most important difference is that the spatial data in IP networks is not only correlated to other spatial data at the same point in time , but also able to directly influence the future realizations of correlated locations with strict order ( i.e. , the Spatio-Temporal Induction effect as we will disclose later ) . Moreover , these existing studies only target at a coarse-grained timescale above minutes or even hours . Models at a sub-second granularity , as FlowNN functions , require deeply combining the spatio-temporal data trends , as well as the system structural knowledge and working mechanism . 1.3 OUR CONTRIBUTIONS . We claim two critical contributions : 1 ) we formulate the crucial traffic flow modelling problem in IP networks as the representation learning task in deep learning , and develop a customized neural network–FlowNN and the associated Induction Operation to extract the domain-specific spatiotemporal data correlations in IP traffic flows . To the best of our knowledge , this is the first work to design customized neural network and learning strategy by deeply combining the IP network structure and working mechanism . The Induction Operation also makes it the first time for a data-driven learning model able to infer the data features at a millisecond granularity , which is usually treated as the ‘ noise ’ region by existing coarse-grained models ; 2 ) we report the state-of-the-art performance over the baselines in different type of practical network applications , which provides a good testament of our model . 2 SPATIO-TEMPORAL INDUCTION . By stacking the networking feature timeseries sampled at all the nodes that a flow passes through , the IP traffic flow data can be organized in the form of high-dimensional tensor time series , as shown in Fig . 3a . The feature ( denoted as xtf , n ) could be the average flow rate ( i.e. , the amount of packet bits received in each unit measurement time ) at each node or the average per-hop packet delay etc . The routing path herein constitutes the most significant attribute for the generative process of each IP traffic flow . This creates many peculiar properties in such flow data . For example , for a flow with a routing path [ 1→4→12 ] in Fig . 3a , the current data at node 4 was originated from the history data at its predecessor node 1 , but delayed by at least2 the link delay ∆t . These data will also flow to its successor node after certain delays . This shows that the flow state at a node is physically 2Packet processing and queueing will also impose extra delay . driven3 by the past flow state at its predecessor node . Therefore , such time-resolved flow data not only tells us who is related to whom but also when and in which order relations occur . This forms the S-shaped data correlation pattern , as exemplified in Fig . 3a . An accurate modelling of the flow data requires attention to such domain-specific data properties , which are absent in the majority of existing learning models if not all of them . Fig . 3b plots a sample of flow traces at two neighboring path nodes from the WIDE dataset . We can observe that an exceeding of data rate at node 1 over node 2 for some duration ( e.g. , T1 ) will always induce a subsequent duration T2 when the data rate at node 2 is greater than that at node 1 . Moreover , the cumulative amount of data bits the two nodes forward in such two durations are almost same , as indicated by the rate difference between the two nodes at the bottom of Fig . 3b . This illustrates that there is a stronger correlation among the data in these two durations , and the future realizations at T2 are subject to the constraint of the states at T1 by local flow conservation4 between the two nodes . In analogy to the concept of Electromagnetic Induction5 in Physics , in what follows , we introduce the Spatio-Temporal Induction ( STI ) effect in IP traffic flows . Accordingly , a learning model is proposed to model the network traffic flows at a fine-grained timescale . Definition 1 . Spatio-Temporal Induction is the production of the temporal evolutions of a flow from the history spatial patterns at its correlated locations . The STI builds a physical relationship between the spatial and temporal patterns of IP flow data . This provides a more accurate interpretation to the underlying data generating process than the trending information manifested by the data itself . Such induction effect is created by the IP network structure and working mechanism , which is preserved when the flow data is sampled at any timescale in practice . Next , we develop an Induction Operation to concretely capture the S-shaped data correlation in IP flow data and propose what we call FlowNN to generate the desired traffic model for IP networks .

Summary: The goal of this study is the 1-step prediction of flow rate in flow networks. They first define a “spatial-temporal induction effect (STI)” and claim it to be the universal property of flow networks. Their main contribution is their proposed “flow neural network” which is based on the STI effect and a combination of GCN and GRU architectures.  According to authors, the novelty of their work lies in the fact that they consider the spatiotemporal features of the flow network simultaneously, whereas the previous works only consider them separately.