A novel method for estimating flow length distributions from double-sampled flow statistics
Since the generation of detailed traffic statistics does not scale well with link speed, increasingly passive traffic measurement employs sampling at the packet or flow level. Sampling has become an attractive and scalable means to measure flow data on high-speed links. However, knowing the length distributions of traffic flows passing through a network link is useful for some applications such as inferring traffic demands, characterizing source traffic, and detecting traffic anomalies. Passive traffic measurement increasingly makes inferences from sampled network traffic. Previous work has shown the inaccuracy of estimating flow length distributions from sampled traffic when the sampling is performed at the packet level. Therefore, we employ double-sampling in collecting network traffic statistic and give a novel method that uses flow statistics formed from double-sampled packet stream to infer the absolute frequencies of lengths of flows in the unsampled stream. We achieve this through statistical inference and by exploiting pareto distribution feature. We use scaling method to obtain the distribution of traffic that evaded flow sampling altogether. We use piecewise Pareto distribution to fit the original traffic. The method allows us to recover the complete flow length distribution.
estimation length distribution double sampling ip flows