Quantifying Host-based Application Trafﬁc with
Multi-Scale Gamma Model
Kensuke Fukuda

Kenjiro Cho

Hiroshi Esaki

NII / PRESTO, JSTkensuke@nii.ac.jp

IIJkjc@iijlab.net

The Univ. of Tokyo
hiroshi@wide.ad.jp

frequency

The modeling and characterization of Internet traﬃc isessential for simulations, traﬃc classiﬁcation, anomaly detection and QoS. However, characterization of application
traﬃc should be done from more various perspectives torealize practical utilization of the characteristics. For example, the following question is still open: “what kind ofapplication should be characterized by which model?”.
In this paper, to quantify behaviors of application traﬃc,
we apply the multi-scale gamma statistical model to a massive amount of dataset. This model was originally proposedin [2] for characterizing aggregated traﬃc. It approximates ahistogram of the number of packets during a unit time ∆ asa gamma distribution, which is uniquely determined by twoparameters: α and β. We aim to show the applicability ofthese parameters to application-based methods such as trafﬁc classiﬁcation and anomaly detection. Our preliminaryﬁndings are the following. (1) Application-speciﬁc traﬃchas speciﬁc ranges of α and β. For example, (a) DNS trafﬁc shows α ∈ [0.1, 10] and β ∈ [1, 10], and (b) scan traﬃc
represents α ∈ [1, 50] and β ∈ [0.1, 10]. (2) α and β of legitimate events are generally stable on a link, but are highlyvariable with link upgrades. In contrast, anomaly events arecharacterized by higher α and β.

0.14035302520151050raw datafitting

0.08
0.06
0.04
0.0200300

600

0900

10time [s]2030405060# of packets

Figure 1: An example of gamma model ﬁtting. Left:a traﬃc from a DNS server. Right: the histogramof X∆ (t) and an estimated gamma distribution.
1000

100

beta (scale)

1. INTRODUCTION# of packets

Yosuke Himura
The Univ. of Tokyo
him@hongo.wide.ad.jp

10

1

0.1
0.01

DNS
MAIL
P2P
WWWS
WWWC
FLOOD
SCAN

0.1

110100alpha (shape)

2. CHARACTERIZATION OF APPLICATION
TRAFFIC WITH MULTI-SCALE GAMMAMODELLet ∆ be the time bin to aggregate packets, and let X∆ (t)be the number of packets which arrive during t-th ∆. The
histogram of X∆ (t) is approximated by a gamma distribu` X ´α−1`´1tion fα,β (X) = βΓ(α)
exp − X, where Γ(x) is theββgamma function. α decides the shape of the histogram (e.g.,
the shape is exponential for α ≤ 1), whereas β determines
the scale of the histogram. In this paper, we empirically set∆ = 1s. Fig.1 shows an example of ﬁtting by this model.
The left ﬁgure represents time-series data; x-axis is time t,and y-axis is X∆ (t). The right ﬁgure illustrates the histogram of X∆ (t) (boxes) and the gamma distribution estimated by the moment method (line).
In order to characterize application traﬃc, we need toidentify application traﬃc from aggregated traﬃc. For practical analyses, we do our analysis at the host-level by classifying speciﬁc hosts of representative applications into nine
Copyright is held by the author/owner(s).
PAM2009, April 1-3, 2009, Seoul, Korea.
.

Figure 2: Correlation among application type, α andβ (on 2007-09-16). Application-speciﬁc traﬃc has
speciﬁc ranges of the parameters.

categories. We use classiﬁcation heuristics based on protocol types, TCP ﬂags, port numbers, communication network structures. The nine categories are DNS, MAIL, P2P,
WWWS (www server), WWWC (www client), FLOOD (ﬂooding attack), SCAN (scanning activity), OTHER (other identiﬁed application such as FTP and SSH), UNKNOWN (unidentiﬁed host). For the characterization of application traﬃc,
we use ﬁrst seven categories and concentrate on the hosts
which have more than 1000 packets in a trace for obtaining
reliable statistics.
For the evaluation dataset, we use the MAWI traﬃc traces[1].
These real data are measured on a trans-paciﬁc link between
Japan and U.S. everyday from 2001. These data are composed of 15 minutes pcap traces (from 14:00-15:00, JST).
The link was upgraded two times: from 18 Mbps to 100
Mbps (in June and July 2006), and from 100 Mbps to 150
MBps (in June 2007).

DNS
10

alpha

8

FLOOD

WWWC

MAIL

P2PSCAN

WWWS(a)642
2002-01-01

2003-01-01

140 (b)120
100806040200
2001-01-01

2002-01-01

2003-01-01

2004-01-01

2005-01-01

2006-01-01

2004-01-01

2005-01-01

2006-01-01

2007-01-01

2008-01-01

2009-01-01

2007-01-01

2008-01-01

2009-01-01

beta

0
2001-01-01

18 Mbps

100 Mbps

date150 Mbps

Figure 3: Time evolution of α and β (from 2001-01-01 to 2008-12-31). Each points has aggregated averagevalue for 1 month and includes its standard error.

3. RESULTS
Correlation between application type and the parameters: Fig.2 illustrates the correlation among application type, α and β with a trace from 2007-09-16. The x-axis
is α, the y-axis is β, and each symbol stands for corresponding type of application traﬃc. This ﬁgure indicates thattype of application traﬃc depends on α and β, so the diﬀerence in application traﬃc is characterized by the multi-scale
gamma model. For example, SCAN symbols are located onα ∈ [1, 50] and β ∈ [0.1, 10]. Thus, the scanning activity
produces Gaussian and low-scale histogram of X∆ (t). From
time-series perspective, the scanner hosts continuously andconstantly send packets. For another example, DNS symbols are concentrated on α ∈ [0.1, 10] and β ∈ [1, 10]. So,
DNS represents various shape and scale of the histogramin this time-scale. On the other hand, the WWW hosts(WWWC and WWWS) produce the wide range of α and β.
A plausible explanation of this eﬀect is a mixture of interactive ﬁle-transfer and video streaming in the same protocol(HTTP). To summarize, α and β can be used as features for
identiﬁcation of malicious outlier traﬃc and classiﬁcation of
application traﬃc.
Time evolution of the parameters: Fig.3 representsthe time evolution of α and β from 2001-01-01 to 2008-1231. For each symbol (application traﬃc), the x-axis is date
and the y-axis is the average value of the parameter whichincludes its standard error (s.e.) at the aggregation level =1 month. When the link bandwidth was 18 Mbps, α andβ are generally stable except for P2P, SCAN and FLOOD.
P2P shows decreasing α, which indicates that the shape ofthe histogram changes from Gaussian to exponential; Thisimplies changes in the dominant form of P2P software (e.g.,from hybrid P2P to pure P2P, or from ﬁle transfer to video
streaming), and does not imply upgrades of other links on
the path because traditional applications (e.g., DNS) show
stable α. Conversely, SCAN and FLOOD produce higher
α and β. One reason of these higher values is massive
outbreaks of worms; For example, the massive outbreak of
Sasser worm (from June 2004 to June 2005) obviously affected β. Thus, we can detect anomaly traﬃc by observing

α and β. On the other hand, when the 18 Mbps link wasreplaced with 100 Mbps one, β highly increased. Since thehigher packet processing speed makes X∆ (t) highly variable,
the scale of the histogram should be enlarged. In contrast,
β of DNS was not aﬀected by the bandwidth. Since DNS
hosts frequently send answer packets and do not transfer
large-size ﬁles, the hosts constantly and continuously sends
a certain amount of packets, so the histogram of X∆ (t) is
independent of bandwidth. In summary, α and β of legitimate events are mainly stable on a link, and become higher
according to the bandwidth upgrade. Anomaly events represent high and variable α and β, and we can detect anomaly
traﬃc by observing them.

4.

SUMMARY

In this paper, we characterized application traﬃc by themulti-scale gamma model, and we obtained the following
ﬁndings: (1) Application-speciﬁc traﬃc has speciﬁc rangesof α and β. (2) α and β of legitimate events are generally
stable on a link, but are highly variable with link bandwidth
upgrades. In contrast, anomaly events are characterized byhigher α and β.
The goal of our study is to discover the characteristics
which quantify diﬀerences in application traﬃc, and to show
the applicability of the characteristics to application-based
methods such as traﬃc classiﬁcation and anomaly detection.
In the future, we will improve our classiﬁcation method formore detail and appropriate characterization and use traﬃc
traces from other links, investigating the eﬀects of ∆.

5.

REFERENCES[1] K. Cho, K. Mitsuya, and A. Kato. Traﬃc Data
Reposity at the WIDE Project. USENIX 2000
FREENIX Track, June 2000.
[2] A. Scherre, N. Larrieu, P. Owezarski, P. Borgnat, andP. Abry. Non-Gaussian and Long Memory Statistical
Characterisations for Internet Traﬃc with Anomalies.
IEEE Transactions on Dependable and Secure
Computing, vol. 4, no. 1, pages 56–70, October 2006.