The Impact and Implications of the Growth
in Residential User-to-User TrafﬁcKenjiro Cho

Kensuke Fukuda

Hiroshi Esaki

Akira Kato

IIJ

NII

Univ. of Tokyo

Univ. of Tokyo

ABSTRACT

1.

It has been reported worldwide that peer-to-peer traﬃc istaking up a signiﬁcant portion of backbone networks. In particular, it is prominent in Japan because of the high penetration rate of ﬁber-based broadband access. In this paper, weﬁrst report aggregated traﬃc measurements collected over
21 months from seven ISPs covering 42% of the Japanese
backbone traﬃc. The backbone is dominated by symmetricresidential traﬃc which increased 37% in 2005. We further
investigate residential per-customer traﬃc in one of the ISPs
by comparing DSL and ﬁber users, heavy-hitters and normal
users, and geographic traﬃc matrices. The results revealthat a small segment of users dictate the overall behavior;
4% of heavy-hitters account for 75% of the inbound volume,and the ﬁber users account for 86% of the inbound volume.
About 63% of the total residential volume is user-to-user
traﬃc. The dominant applications exhibit poor locality andcommunicate with a wide range and number of peers. The
distribution of heavy-hitters is heavy-tailed without a clear
boundary between heavy-hitters and normal users, whichsuggests that users start playing with peer-to-peer applications, become heavy-hitters, and eventually shift from DSL
to ﬁber. We provide conclusive empirical evidence from alarge and diverse set of commercial backbone data that the
emergence of new attractive applications has drastically affected traﬃc usage and capacity engineering requirements.

Over the past few years, an unprecedented increase inuser-to-user traﬃc has been observed worldwide, particularly in Japan due to its high penetration rate of ﬁber-based
broadband access. Figure 1 depicts the traﬃc growth inJapanese backbones in terms of aggregated peak traﬃc atmajor IXes: JPIX[11], JPNAP[12], and NSPIXP[18].

INTRODUCTION

150

8
Traffic volume
Growth rate

7
6

Annual growth rate

Aggregated IX traffic [Gbps]

200

5
100

4
3

50

2
1

0

2000 2001 2002 2003 2004 2005 2006
Year

0

Figure 1: Traﬃc growth of the aggregated peak rate
at the major Japanese IXes
Although a large part of the traﬃc increase on commercial
backbones is often attributed to peer-to-peer traﬃc, there
is little work in literature with statistics detailed enough toprove it. It is also diﬃcult to plan for the future because
residential access and its traﬃc are undergoing a transformation; new innovations in access networking technologies
continue to be developed, and new applications as well as
new usage of older applications are emerging to take advantage of low-cost high-speed connectivity. Japan leads othercountries by far in Fiber-To-The-Home (FTTH) penetration[20], where the number of FTTH subscribers is increasing exponentially while the increase in DSL subscribers is slowing
down as shown in Figure 2 [29]. (FTTH includes Fiber-ToThe-Building with VDSL for in-building wiring.)
There is a strong concern that, if this trend continues,
Internet backbone technologies will not be able to keep up
with the rapidly-growing residential traﬃc. Moreover, commercial ISPs will not be able to invest in backbone networks
simply for supporting this low-proﬁt customer segment.
In order to ensure the evolution of the Internet, it is essential that we understand the eﬀects of growing residential
traﬃc, but it is diﬃcult both technically and politically toobtain traﬃc data from commercial ISPs. Most ISPs collecttraﬃc data which contains sensitive information, and thus
seldom make it available to others. In addition, measure-

Categories and Subject Descriptors
C.2.3 [Computer-Communication Networks]: Network
Operations—Network monitoring

General Terms
Measurement, Management

Keywords
ISP backbone traﬃc, Residential broadband, Traﬃc growth

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for proﬁt or commercial advantage and that copies
bear this notice and the full citation on the ﬁrst page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior speciﬁc
permission and/or a fee.
SIGCOMM’06, September 11–15, 2006, Pisa, Italy.
Copyright 2006 ACM 1-59593-308-5/06/0009 ...$5.00.

1Number of subscribers [million]

16
12

(B1)
external 6IXes
JPNAP/JPIX/NSPIXP

DSL
CATV
FTTH

(B3)
external international

external edge

8

ISP

4
0

(B2)
external domestic
local IXes
private peering/transit

customer edge

2000 2001 2002 2003 2004 2005 2006
Year

(A1)
RBB customers
DSL/CATV/FTTH(C) prefectural

Figure 2: Increase of residential broadband subscribers in Japan: 23.3 million total broadband subscribers, 14.5 million for DSL, 3.3 million for CATV,and 5.5 million for FTTH as of March 2006.

(A2)
non-RBB customers
leased lines
data centersdialupFigure 3: Five traﬃc groups for data collection at
ISP customer and external boundaries

ment methods and policies diﬀer from ISP to ISP so that it
is in general not possible to compare a data set with anotherset obtained from a diﬀerent ISP.
Seeking a practical way to investigate the impact of residential broadband traﬃc on commercial backbone networks,
we formed a study group with specialists including members from seven major Japanese commercial ISPs in order
to identify the macro-level impact of residential broadband
traﬃc on ISP backbones [6]. Our goal is to obtain a clearer
grasp of the ratio of residential broadband traﬃc to othertraﬃc, changes in traﬃc patterns, and regional diﬀerences
across ISPs.
We have collected aggregated bandwidth usage logs forseveral categories of traﬃc. The results reveal that the backbone is dominated by symmetric residential traﬃc which
increased 37% in 2005. The peak hours have shifted fromoﬃce hours to evening hours, and a considerable amount oftraﬃc is constantly ﬂowing.
Using these statistics as reference points, we have performed further analyses of residential traﬃc data provided
by one of the ISPs. The results reveal surprisingly diverse
behavior of residential traﬃc.

residential broadband traﬃc can be analyzed. The data setsshould be aggregated with those provided by other ISPs sothat the share of each ISP is not revealed.
Our focus is on traﬃc crossing ISP boundaries which canbe roughly divided into customer traﬃc, and external trafﬁc such as peering and transit. For practical purposes, weselected the ﬁve traﬃc groups in Figure 3 for data collection.
(A1) RBB customers represent residential broadband customer lines. This group also includes small business customers
using residential broadband access.
(A2) non-RBB customers represent customer lines other than
RBB customers, including leased lines, data centers, anddialup lines. This group includes RBB customers behindleased lines, e.g., second or third level ISPs, since ISPs do
not distinguish them from other leased lines.
(B1) external 6IXes represent links for 6 major IXes, namely
JPIX, JPNAP and NSPIXP in both Tokyo and Osaka inorder to compare measurements at these IXes as well as toknow the traﬃc share of our measurement.
(B2) external domestic represents domestic external links other than the 6IXes, including regional IXes, private peeringand transit. We used the term “domestic” to indicate thatboth ends of a link are located in Japan. This group also
includes domestic peering with global ASes.

2. DATA COLLECTION
Our data sets were collected using two diﬀerent methods.
The ﬁrst set was collected by aggregating interface countersof edge routers from seven ISPs for analysis of residential
traﬃc at a macro-scopic level. The other set was collectedby Sampled NetFlow [2] from one of the ISPs for detailed
per-customer analysis.

(B3) external international represents international externallinks with one end point outside of Japan.
(C) prefectural represents RBB links categorized into 47 prefectures in Japan. This group is a subset of (A1), and covers
two major residential broadband carriers who provide aggregated links per prefecture to ISPs. Other RBB carriers
without prefectural information are not used for this group.

2.1 Data Collection of Aggregated Trafﬁc

It is impossible to draw a strict line for grouping, e.g., residential/business and domestic/international, on the global
Internet, so these groups are chosen by the existing operational practices of the participating ISPs. We re-aggregate
each ISP’s aggregated logs, and only the resulting aggregated traﬃc is used in our study so as to not reveal theshare of each ISP.
Our main focus is on (A1), RBB customers, but we examine the other categories to understand the relative volumeof (A1) with respect to other types of traﬃc as well as tocross-check the correctness of our results. (A2), non-RBB
customers, is used to obtain the ratio of residential broadband traﬃc to total customer traﬃc. The total customer

We found that most ISPs collect interface counter valuesof almost all routers in their service networks via SNMP,
and archive per-interface traﬃc logs using MRTG [22] or
RRDtool [21]. Thus, it is possible for the ISPs to provide
aggregated traﬃc information if they can classify router interfaces into a common set.
There are several requirements in order to solicit ISPs todivulge traﬃc information. We need to ﬁnd a common data
set which all the participating ISPs are able to provide withmoderate workload and investment. The data set should
be coarse enough not to reveal sensitive information about
the ISP but be meaningful enough so that the behavior of2

traﬃc (A) is (A) = (A1) + (A2). (B1), external 6IXes, and(B2), external domestic, are used to estimate the coverageof the collected data sets. (B3), external international, isused to compare domestic traﬃc with international traﬃc.
The total external traﬃc (B) is (B) = (B1) + (B2) + (B3).
(C), prefectural, is used to measure regional diﬀerences.
In general, it is meaningless to simply sum up traﬃc values from multiple ISPs since a packet could cross ISP boundaries multiple times. Customer traﬃc is, however, summablebecause a packet crosses customer edges only once in eachdirection, when entering the source ISP and exiting the destination ISP. The numbers for external traﬃc are overestimated since a packet could be counted multiple times if
it travels across intermediate ISPs other than ingress andegress ISPs. However, the error should be negligible in thisparticular result since the ISPs in our data sets are peering,and thus, not providing transit to each other.
We collected month-long traﬃc logs from the participating
ISPs. The collected logs have a time resolution of two hours
since it was the highest common factor for month-long data.
This is because both MRTG and RRDtool aggregate oldrecords into coarser records in order to bound the databasesize. In MRTG, 2-hour resolution records are maintainedfor 31 days in order to draw monthly graphs. RRDtool does
not have ﬁxed aggregation intervals but it is most likelythat RRDtool is conﬁgured to maintain 1-hour or 2-hourresolution records needed for monthly graphs. Although thepeak rate is often used for operational purposes, only themean rate is collected since the peak rate is not summable.
We developed a perl script to read a list of MRTG and
RRDtool log ﬁles, and to aggregate traﬃc measurements fora given period at a given resolution. It outputs “timestamp,
in-rate, out-rate” for each time step. Another script produces a graph using RRDtool. We provided the tools to the
ISPs so that each ISP could create aggregated logs by themselves. This allows ISPs not to disclose the internal structureof their network or unneeded details of their traﬃc.
The highest workload for the ISPs is to classify a large
number of per-interface traﬃc logs and create a log list foreach group. For large ISPs, the number of existing perinterface traﬃc logs can exceed 100,000. To reduce theworkload, ISPs are allowed to use the internal interface of aborder router instead of a set of external (edge) interfaces
if the traﬃc on the internal interface is an approximation of
the sum of the external interfaces. In this case, we instruct
the tool to swap “in” and “out” records since the notation in
the per-interface logs depicts the perspective of the routers
but inbound/outbound records in our data sets signify the
ISPs’ point of view.
We analyzed month-long traﬃc logs from seven major
Japanese ISPs six times over 21 months; September, October, November in 2004, May and November in 2005, andMay in 2006. For data analysis, we focus on the data sets
in 2005 to compare them with the per-customer data eventhough the latest set is from May 2006. We collected thedata separately for each month, and checked consistency
in each ISP’s share, diﬀerences from the previous measurements, the coverage of the IX traﬃc, and others. We also
provide the aggregated results to the ISPs so that each ISP
can compare and check its own data against the aggregatedresults. Thus, we are fairly conﬁdent about the accuracyof the results. After the initial trials over four months (theﬁrst results from August 2004 had errors and have neverbeen published), we decided to collect data only twice ayear to reduce the workload of the participating ISPs.
Monthly traﬃc logs with two-hour resolution allow us to
identify major changes in each ISP’s traﬃc. When suchchanges are found, we contact the ISP to conﬁrm the cause of
the change, e.g., a network reconﬁguration, an outage, missing SNMP data, or a mis-classiﬁcation of interface counterlogs. Afterwards, if necessary, we ask the ISP for corrected
data. We indeed found short periods of missing SNMP dataa few times but their impacts to the monthly average wereless than 1%. Once, we found a mis-classiﬁcation and asked
for corrected data.

2.2

Data Collection of Per-Customer Trafﬁc

In order to further analyze the behavior of residential trafﬁc, we obtained Sampled NetFlow data from one of the participating ISPs. This ISP has residential broadband customers over DSL and ﬁber but not over CATV. Data wascollected from all edge routers accommodating residential
broadband customers. The sampling rate used was 1/2048
so as to not overload the routers. We believe it is enoughfor analyzing heavy-hitters but there is a certain amount ofsampling error, especially for lightweight users. The traﬃcvolume is derived by multiplying the measured volume by
the sampling rate.
A week-long data set was collected ﬁve times: April, May,
October in 2004, February and July in 2005. In this paper,
we use only the two sets from February and July 2005 to
focus on traﬃc in 2005.
Data from February 2005 was used to analyze per-customer
behavior in Section 4.1 through 4.3 by matching customer
IDs with the assigned IP addresses. The ISP provided theinbound/outbound traﬃc volume of each customer in one
hour resolution as well as customer’s attributes: the line
type (DSL or ﬁber), and the prefecture.
Data from July 2005 was used to analyze geographic communication patterns from Section 4.4 through 4.5. In ourdata, one end of a ﬂow is always the residential customerof the ISP but the other end is generally a customer of another ISP. Therefore, it is not possible to classify both ends
by the ISP’s information alone. For this reason, we used
two geo-IP databases, Cyber Area Research Inc’s SUTFPOINT and Digital Envoy’s Netacuity, to classify both endsof the ﬂows. The former database maps the address blocksof domestic residential customers to prefectures, but it does
not cover non-residential addresses such as data-centers andleased-lines. The advantage of using this database is that
we can distinguish residential users from other domestic
users. The addresses not covered by the former databaseare classiﬁed simply into domestic and international by thelatter database. Thus, domestic corresponds mainly to datacenters and leased lines in Japan, but it also includes residential address blocks not listed in the former geo-IP database.

3.

ANALYSIS OF AGGREGATED TRAFFIC

The results were obtained by aggregating all traﬃc logs
provided by the seven ISPs. Each ISP provided monthlong traﬃc logs with 2-hour resolution. Both MRTG and
RRDtool compute 2-hour boundaries in UTC so that theboundaries fall on odd hours in Japanese Standard Time
(UTC+9). Throughout the paper, inbound and outboundare presented from the ISPs’ point of view.
3Figure 4: Growth of customer traﬃc: (A1) RBB
customer and (A2) non-RBB customer

Figure 5: Growth of external traﬃc: (B1) 6 major
IXes, (B2) other domestic and (B3) international

3.1 Growth of Trafﬁc

Table 2: Monthly average rates of aggregated external traﬃc

The monthly average rates in bits/second of the traﬃcgroups are shown in Tables 1 through 4. Table 1 shows theaverage rates of aggregated customer traﬃc, and the traﬃc
growth is shown in Figure 4. Between November 2004 andNovember 2005, the growth rate of the RBB customer traﬃc(A1) was 26% for inbound, 46% for outbound, and 37% for
the combined volume. The diﬀerence between inbound andoutbound slightly widened in the ﬁrst 6 months. The data
for non-RBB customer traﬃc was obtained only from thefour ISPs; it is diﬃcult for the other ISPs to distinguish external links from other links due to historical reasons. Since(A2) from these other ISPs is missing, it is not possible todirectly compare (A1) with (A2). Thus, we estimated theratio of (A1) to (A) using only data from the 4 ISPs withboth (A1) and (A2). The estimated ratio (A1)/(A1+A2)
was 59% for inbound and 64% for outbound in November
2005.

2004

2005
2006

2004

2005
2006

Sep
Oct
Nov
May
Nov
May

(B2)ext-dom(7 ISPs)
in
out
48.2G
37.8G
53.1G
41.6G
55.1G
43.3G
73.3G
58.4G
80.9G
68.1G
94.9G
77.6G

(B3)ext-intl(7 ISPs)
in
out
25.3G
14.1G
27.7G
15.4G
28.5G
16.7G
40.1G
24.1G
57.1G
39.8G
68.5G
47.8G

the inbound traﬃc volume for the total external traﬃc (B)should be close to the outbound traﬃc volume for the totalcustomer traﬃc (A). Similarly, the outbound traﬃc volume(B) should be close to the inbound traﬃc volume (A). However, non-RBB customer data is provided by only 4 ISPs. If
we interpolate the missing ISPs in the non-RBB customer
traﬃc using the ratio from the four reporting ISPs, the total
inbound and outbound customer traﬃc for November 2005
is estimated to be 248.4Gbps and 304.4Gbps, respectively.
These ﬁgures are higher than those for the total external
traﬃc, and the diﬀerence is considered to be customer trafﬁc whose source and destination belong to the same ISP.

Table 1: Monthly average rates of aggregated customer traﬃc(A1)customer-RBB(7 ISPs)
inboundoutbound
98.1G
111.8G
108.3G
124.9G
116.0G
133.0G
134.5G
178.3G
146.7G
194.2G
173.0G
226.2G

Sep
Oct
Nov
May
Nov
May

(B1)ext-6ix(7 ISPs)
in
out
35.9G
30.9G
36.3G
31.8G
38.0G
33.0G
47.9G
41.6G
54.0G
48.1G
66.2G
60.1G

(A2)customer-non-RBB(4 ISPs)
inboundoutbound
14.0G
13.6G
15.0G
14.9G
16.2G
15.6G
23.7G
23.9G
36.1G
29.7G
42.9G
38.3G

Table 3: Monthly average rates of total customer
traﬃc and total external traﬃc
2004

Table 2 summarizes the average rates of aggregated external traﬃc, and the traﬃc growth is shown in Figure 5. It isobserved that the total volume of external domestic traﬃc(B2), mainly private peering, exceeds the volume for the six
major IXes (B1). From this result, it would be misleading
to simply rely on data from IXes to estimate and understand nation-wide traﬃc, because a considerable amount oftraﬃc is exchanged by private peering. At the same time, it
is possible that the ratio of private peering is overestimatedfor the rest of the Japanese ISPs because private peering
is usually exercised only between large ISPs. The ratio ofinternational traﬃc to the total external traﬃc was 30% forinbound and 26% for outbound in November 2005.
Table 3 shows a relationship between the total customer
traﬃc (A) and the total external traﬃc (B). If we assume
all inbound traﬃc from other ISPs is destined to customers,

2005
2006

Sep
Oct
Nov
May
Nov
May

(A)customer(A1+A2)
inboundoutbound112.1G
125.4G
123.3G
139.8G132.2G
148.6G
158.2G
202.2G
182.8G
223.9G
215.9G
264.5G(B)external(B1+B2+B3)
inboundoutbound109.4G
82.8G
117.1G
88.8G
121.6G
93.0G
161.3G
124.1G
192.0G
156.0G
229.6G
185.5G

Last, we examined the relationship between our IX traﬃcdata (B1) and the total input rate of the six major IXes,
as obtained directly from these IXes [29]. In comparison
with the published total incoming traﬃc of these IXes, ourdata consistently represents about 42% of the total traﬃc
as shown in Table 4. If we assume this ratio to be the trafﬁc share of the seven ISPs, the total amount of residential
broadband traﬃc in Japan in November 2005 can roughly be
estimated to be 353Gbps for inbound and 468Gbps for outbound. We are not aware of any data from other countries
against which to compare these numbers.
4

Table 4: IX traﬃc observed at ISPs and IXes
2004

2005
2006

Sep
Oct
Nov
May
Nov
May

(B1)ext-6ix
outbound
30.9G
31.8G
33.0G
41.6G
48.1G
60.1G

6 major IXes
inbound
74.5G
77.1G
80.3G
99.1G
115.9G
139.2G

ratio (%)
41.5
41.2
41.1
42.0
41.5
43.2

Figure 7: Growth of inbound traﬃc of RBB customers between November 2004 and November 2005

3.2 Customer Trafﬁc
Figure 6 shows weekly customer traﬃc. For weekly data
analysis, we took the averages of the same weekdays in amonth. We excluded holidays from the weekly analysis sinceholiday traﬃc patterns are closer to those of weekends.

also observe oﬃce hour traﬃc (from 8:00 to 18:00) in thedaytime on weekdays but traditional oﬃce customer traﬃc
is smaller than residential customer traﬃc. The traﬃc patterns common to both graphs in Figure 6 are diﬀerent fromwell-known academic or business usage patterns in which
the peak is found during oﬃce hours [4, 5, 25].

3.3

External Trafﬁc

The external traﬃc groups are used to understand thetotal traﬃc volume in backbone networks. The top graph in
Figure 8 shows traﬃc to and from the six major IXes (B1).
It is apparent that the traﬃc behavior is strongly aﬀectedby residential traﬃc.

Figure 6: Aggregated weekly traﬃc of RBB customers (top) and non-RBB customers (bottom) in
November 2005. Darker vertical lines indicate thestart of the day (0:00 am in local-time).

The top graph shows RBB customers, (A1), consisting of
DSL/FTTH/CATV residential users. The residential broadband customer traﬃc already exceeds 260Gbps in evening
hours. The inbound and outbound traﬃc volumes are almost equal, and about 120Gbps is constantly ﬂowing inboth directions, probably due to peer-to-peer applications
which generate traﬃc independent of daily user activities.
The diurnal pattern indicates that home user traﬃc is dominant, i.e., the traﬃc increases in the evening, and the peak
hours are from 21:00 to 23:00. Weekends can be identiﬁedby larger daytime traﬃc although the peak rates are similar
to weekdays’. The outbound traﬃc to customers is slightly
larger than the inbound, even though it is often assumed
that home users’ downstream traﬃc is much larger than upstream. We believe that peer-to-peer applications contributesigniﬁcantly to the upstream traﬃc as we will see in Section
4. Figure 7 compares the RBB customer inbound traﬃc in
November 2004 and November 2005. The overall increase
appears to be derived from the growth of the constantly
ﬂowing traﬃc.
The bottom graph in Figure 6 shows the weekly traﬃcof non-RBB customers (A2). Since this group also includes
leased lines used to accommodate second or third level ISPs,
the traﬃc pattern still appears to be dominated by residential traﬃc, which is indicated by the peak hours and thediﬀerences between weekdays and weekends. However, weFigure 8: Aggregated weekly traﬃc for the 6 major IXes (top), other domestic (middle) and international (bottom) in November 2005
The middle graph in Figure 8 shows the external domestic traﬃc (B2) including regional IXes, private peering andtransit but not including traﬃc for the six major IXes. The
traﬃc pattern is very similar to the top graph.
The bottom graph in Figure 8 shows international traﬃc(B3). The inbound traﬃc is much larger than the outbound,and the traﬃc pattern is clearly diﬀerent from the domestic
traﬃc. The peak hours are still in the evening, but outbound traﬃc volume is virtually ﬂat when compared to theinbound volume, suggesting that the traditional behavior of5

3.0

Japanese users downloading content from overseas is stilla non-negligible part of international traﬃc. At the sametime, the constant part is about 70% of the average inboundrate, which could be due to machine-generated traﬃc (e.g.,
peer-to-peer ﬁle-sharing).

Traffic [Gbps]

2.5

3.4 Prefectural Trafﬁc
In order to investigate regional diﬀerences, i.e., between
metropolitan and rural areas, we collected regional traﬃcdata of the 47 prefectures. Figure 9 illustrates aggregatedtraﬃc of one metropolitan prefecture (top) and of one ruralprefecture (bottom). Both graphs exhibit similar temporal
patterns such as peak positions and weekday/weekend behavior. In addition, about 70% of the average traﬃc is constant regardless of the traﬃc volume. These characteristics
are common to other prefectures. One noticeable diﬀerenceis that metropolitan prefectures experience larger volumesof oﬃce hour traﬃc, probably due to larger business usage.

2.0
1.5
1.0
In
Out

0.5
0.0
0

10000
5000
Population [x 1000]

Figure 10: Prefectural traﬃc volumes are roughly
linear to populations
1
in
out

Cumulative traffic

0.8

0.6

0.4
0.2
0 -6
10

-5

10

-4

-3

-2

10
10
10
Cumulative heavy hitters

-1

10

0

10

Figure 11: Cumulative distribution of traﬃc volumeof heavy-hitters in decreasing order of volume
Figure 9: Example prefectural traﬃc: a metropolitan prefecture (top) and a rural prefecture (bottom)
DSL is slightly higher than ﬁber, but there are more heavyhitters among ﬁber users.
Figure 10 plots traﬃc volumes and populations for the 47
prefectures. We found that a prefecture’s traﬃc is roughly
proportional to the population of the prefecture. The result indicates that there is no clear regional concentrationof heavy-hitters of the Internet. That is, the probabilityof ﬁnding a heavy-hitter in a given population is constantand the distribution of aggregated traﬃc volume directly
depends on the population. A possible reason is the availability of fairly universal access services in Japan; 100Mbps
ﬁber access is available in most areas.

Table 5: Ratio of ﬁber and DSL active users in the
February 2005 data set
total
ﬁber
DSL

4.1

ratio (%)
100
46.45
53.55

≥ 2.5GB/day (%)
4.46
3.66
0.80

< 2.5GB/day (%)
95.54
42.79
52.75

Distribution of Heavy-hitters

Figure 11 shows the cumulative distribution of the total
traﬃc volume of heavy-hitters in decreasing order of volume.
The distribution is computed independently for inbound andoutbound traﬃc. The graph reveals a skewed traﬃc distribution among users; the top N% of heavy-hitters use X% of
the total traﬃc. For example, the top 4% use 75% of thetotal inbound traﬃc, and 60% of the outbound. In otherwords, a small group of heavy-hitters represent a signiﬁcantpart of the total traﬃc.
Figure 12 shows the (complementary) cumulative distribution of daily traﬃc per user on a log-log scale, and compares the total users (top) with the ﬁber users (middle) andthe DSL users (bottom). The daily traﬃc volume is theaverage for the week, and the distribution is computed independently for inbound and outbound traﬃc.

4. ANALYSIS OF PER-CUSTOMERTRAFFICThis section analyzes Sampled NetFlow data from one of
the ISPs. By comparing the aggregated traﬃc graphs in theprevious section with the ISP’s corresponding graphs, we
can say that the traﬃc characteristics are consistent. Thus,
although the data sets are from only one ISP, the results arelikely to represent Japanese residential traﬃc.
Table 5 shows the number of unique active users identiﬁed
by customer IDs in the February data set. As we explain
later, users are classiﬁed into two groups by average daily
inbound traﬃc, one of more than 2.5 GB/day and the otherof less than 2.5GB/day. The total number of active users of
6

10

0

10

-1

10

10

-2

10

-3

10

10

10

-5

in
out

10

-6

4

10

5

10

6

7

8

9

10

10
10
10
10
10
Daily traffic per user [bytes]

10

10

10

-1

in
out
4

10

5

0

6

7

8

9

10

10

11

6

7

8

9

10

10

11

10
10
10
10
10
Daily traffic per user [bytes]-1

10

Cumulative distribution

Cumulative distribution

-5

-6

10

-2

10

-3

10

-2

10

-4

10

-5

in
out

10

-6

10

-4

10

11

0

10

-3

10

10

10

-2

10

-4

10

-1

Cumulative distribution

Cumulative distribution

10

0

4

10

5

10

0

-3

-4
-5

in
out

10

6

7

8

9

10

10
10
10
10
10
Daily traffic per user [bytes]

10

-6

10

11

10

4

10

5

10
10
10
10
10
Daily traffic per user [bytes]

10

Figure 13: Cumulative distribution of daily traﬃc
per user: a metropolitan prefecture (top) and a rural prefecture (bottom)-1

Cumulative distribution

10

10

-2
-3

10

10

-4
-5

-6

10

is not found in the plots denoting that there is no signiﬁcant
gap in traﬃc usage among users.
As for prefectural diﬀerences, the distributions look similar across diﬀerent prefectures as shown in Figure 13 which
compares one metropolitan prefecture (top) with one ruralprefecture (bottom). One diﬀerence is the tail length due tothe diﬀerence in the number of users. Another diﬀerence isthat the distribution of the metropolitan prefecture is closer
to that of the total users, and the distribution of the ruralprefecture is closer to that of the DSL users. The results indicate that the distribution of heavy-hitters is similar acrossdiﬀerent regions with slight diﬀerences in the ratio of heavyhitter population due to the ratio of ﬁber users with a larger
heavy-hitter population.

in
out

10

4

10

5

10

6

7

8

9

10

10
10
10
10
10
Daily traffic per user [bytes]

10

11

Figure 12: Cumulative distribution of daily trafﬁc per user: total users (top), ﬁber users (middle)
and DSL users (bottom). The lines are drawn at
2.5GB/day and the top 4% heavy-hitters, the kneeof the total users’ slope.

The distributions is heavy-tailed but there is a knee inthe slope, at the top 4% of heavy-hitters using more than
2.5GB/day (or 230kbits/sec) for the total users, and at the
top 10% using more than 2.5GB/day for the ﬁber users. It isless clear for the DSL users, but a knee can be seen at aroundthe top 2% using more than 2.5GB/day. The distribution
also shows that outbound traﬃc is larger for the majorityof the users on the left side of the knee but it does not holdfor heavy-hitters on the right side of the knee.
The distribution has a diﬀerent slope for those who upload
more than 2.5GB/day so we use this ﬁgure to statisticallydistinguish heavy-hitters from the rest of the users. We classify users who upload more than an average of 2.5GB/day
to be in the heavy-hitter group, and those who upload lessthan 2.5GB/day to be in the normal user group. The normal user group should be interpreted as users other thanthe most inﬂuential heavy-hitters. Note that the diﬀerenceis only in the slope of the distribution, and the boundary
between the two groups is not clear. In other words, users
are distributed statistically over a wide traﬃc volume range,even up to the most extreme heavy-hitters. A concave curve

4.2

Correlation of Inbound and OutboundVolumes

The correlation between inbound and outbound volumes
for each user is shown as log-log scatter plots in Figure 14.
These are taken from a metropolitan prefecture and plot
about 4300 points for ﬁber and about 5400 for DSL but thecharacteristics are common to all the prefectures.
There is a positive correlation as expected, and the highest
density cluster is below and parallel to the unity line where
outbound volume (downstreaming for users) is about ten
times larger than that of inbound. In a higher volume region,a diﬀerent cluster appears to exist around the unity line.
The slope of the cluster seems to be slightly larger than 1,
which explains the inversion of inbound and outbound trafﬁc volumes in Figure 12. It can be also observed that, acrossthe entire traﬃc volume range, the inbound/outbound trafﬁc ratio varies greatly, up to 4 orders of magnitude.
Both ﬁber and DSL plots show similar distributions but,
as expected, the high-volume cluster is larger in the ﬁber
7

11

Daily inbound traffic [bytes]

10

10

10

9

10

8

10

7

10

6

10

5

10

4

10 4
10

6

7

8

9

10

10

6

7

8

9

10

10

10

5

10
10
10
10
10
Daily outbound traffic [bytes]

5

10
10
10
10
10
Daily outbound traffic [bytes]

11

11

Daily inbound traffic [bytes]

10

10

10

9

10

8

10

7

10

6

10

5

10

4

10 4
10

10

11

Figure 15: Fiber weekly traﬃc: total ﬁber users(top), heavy-hitters (middle) and normal users (bottom)

Figure 14: Correlation of inbound and outboundtraﬃc volumes per user in one metropolitan prefecture: ﬁber (top) and DSL (bottom)

are normalized to the peak value of the total active users.
The number of active users is fairly constant for the heavyhitters, especially for DSL. The constant portion seems to
be users running automated data-transfer software. The increase of active users in the morning is larger than that oftraﬃc volume but the increase in the evening is smaller,
which suggests that bandwidth use is more intense, i.e.,higher bandwidth demand per user, in the evening.

plot, especially above the unity line. A plausible interpretation of excess upstream traﬃc of the ﬁber heavy-hitters isthat available bandwidth in ﬁber access is used to compensate for the shortage of upstream bandwidth of DSL heavyhitters. It is also noticeable that there are much more lowvolume users in the DSL plot. However, the boundary of thetwo clusters is not very clear. There seems to be no clear
qualitative diﬀerence in the behaviors of ﬁber and DSL users
except the percentage of heavy-hitters.

4.4

Protocol and Port Usage

Table 6: Protocol breakdown: TCP dynamic ports
account for 83% of the total traﬃc

4.3 Temporal Behavior
Figure 15 and Figure 16 compare the temporal behaviorsof the ﬁber users and the DSL users. The volume is normalized to the peak value of the total traﬃc size so as to notreveal the absolute traﬃc volume of the ISP. The graphs areshown in the same scale to compare ﬁber and DSL volumes.
The plots show that the inbound and outbound volumes
are almost equal for ﬁber traﬃc but the inbound is 61%
larger for heavy-hitters and the outbound is 166% larger forthe normal users. The total is counterbalanced by the twogroups. In the DSL traﬃc, the outbound volume is 83%
larger for the total users, only 11% larger for the heavyhitters and 179% larger for the normal users. The total
reﬂects the oﬀset of the normal users.
The inbound traﬃc of the ﬁber heavy-hitters is muchlarger than the outbound traﬃc, and has large daily ﬂuctuations. On the other hand, the inbound traﬃc of DSL
heavy-hitters is saturated. As a result, the ﬁber traﬃc accounts for 86% of the total inbound volume and 80% of thetotal residential volume, and the behavior of the total traﬃcis heavily inﬂuenced by the ﬁber heavy-hitters.
Figure 17 compares the temporal change in the number ofactive users in ﬁber and DSL. Again, the active user numbers

protocol
TCP

UDP
ESP
GRE
ICMP
othersport name(%) port name
(%)
*
97.43
(< 1024
13.99)
81 0.1580 http
9.32
25 smtp
0.14
0.13
20 ftp-data
0.93 119 nntp
21 ftp0.11
554 rtsp
0.38
443 https
0.30
22 ssh0.09
110 pop3
0.17
others2.27(>= 1024
83.44) 1935 macromedia-fsc 0.20
6699 winmx
1.40 1755 ms-streaming
0.20
0.13
6346 gnutella
0.92 2265 7743 winny
0.48 1234 0.126881 bittorrent
0.25 4662 edonkey
0.12
others
79.41
6348 gnutella
0.21*
1.38 6257 winmx0.06
6346 gnutella
0.39
others
0.93
1.09
0.07
0.01
0.02

Table 6 shows the ranking of protocols and ports. To
rank port numbers in TCP and UDP, we took the smallerof the source and destination ports for a ﬂow. TCP ports arefurther divided into well-known ports that are smaller than1024, and dynamic ports that are equal to or larger than8

Table 7 shows the traﬃc matrix among residential users(RBB), domestic data-centers and leased-lines (DOM), andinternational addresses (INTL). Note that the data covers
only the ISP’s residential customer traﬃc so that the results
depict only the behavior of residential users. Residentialuser-to-user traﬃc accounts for 63% of the total residential
traﬃc. This is a conservative estimate since the international group also includes residential users.
Table 7: Traﬃc matrix of the July data set
src\dst
ALL
RBB
DOM
INTL

ALL
100.0
77.0
18.0
5.0

RBB
84.8
63.3
16.7
4.8

DOM
11.1
9.8
1.1
0.2

INTL
4.1
3.9
0.2
0.0

Table 8: Traﬃc matrices further divided into heavyhitters and normal users
heavy-hittersnormal users
src\dst ALL RBB DOM INTL ALL RBB DOM INTL
ALL
69.7 57.8
8.6
3.3 30.3 27.0
2.6
0.7
RBB
59.4 48.4
7.8
3.2 17.6 14.9
2.0
0.7
DOM
8.7
7.9
0.7
0.1 9.3
8.8
0.5
0.0
INTL
1.6
1.5
0.1
0.0 3.4
3.3
0.1
0.0

Figure 16: DSL weekly traﬃc: total DSL users(top), heavy-hitters (middle) and normal users (bottom)

A surprisingly large portion, about 90%, is domestic communication where both ends are either domestic residential
users or other domestic addresses. One possible explanationis language and cultural barriers; the majority of content
is in the Japanese language and/or is popular only with
the Japanese. However, there are many Japanese worldwide
who may access content in Japan, and Japanese content suchas animation is popular with non-Japanese as well. Anotherplausible explanation is that domestic ﬁber users are connected so well in terms of bandwidth and latency that supernodes in peer-to-peer networks are interconnected mainly
among domestic heavy-hitters.
A small degree of mis-classiﬁcation is found in the table; 1.5% among DOM and INTL. Since the data is takenfrom residential traﬃc, and non-residential ﬂow entries, e.g.,management ﬂows for routers, were ﬁltered by the ISP in
advance, the traﬃc not including RBB should be zero. The
disparity is caused by new residential address blocks notlisted in the geo-IP database. Although it was possible toﬁx the database using the information from the ISP, we did
not do so since errors of the same kind are expected in address blocks of other ISPs at a similar error rate.
Table 7 is further divided into heavy-hitters and normal
users in Table 8 where ﬂows with a heavy-hitter’s address ateither end are classiﬁed to the heavy-hitter traﬃc and therest of the ﬂows are classiﬁed into the normal user traﬃc.
The ratio of user-to-user traﬃc is 69% (48.4/69.7) for heavyhitters and 49% (14.9/30.3) for normal users. The ratio ofdownload traﬃc from DOM or INTL to RBB is much larger
for the normal users.
To show the geographic distribution of domestic user-touser traﬃc, a prefectural traﬃc matrix is shown in Figure 18
in which the prefectures are ordered by geographic locations
for source (row) and destination (column). In order to observe diﬀerences among prefectures, the traﬃc volumes arenormalized to the source prefecture so that the sum of thecolumns for each row becomes 100%.

1024. We do not distinguish registered ports from dynamic
ports since many implementations use the registered port
range from 1024 through 49151 for dynamic ports.
Port 80 (http) accounts only for 9% of the total traﬃc.
TCP dynamic ports account for 83% but the usage of eachport is small, probably because the most popular peer-topeer ﬁle-sharing software in Japan, WINNY [13], uses arbitrary ports. The largest one, port 6699, is only 1.4%. It
is evident that it is no longer possible to make use of port
numbers for identifying applications.

4.5 Geographic Trafﬁc Matrices
To investigate geographic communication patterns amongresidential users, we classify traﬃc using the geo-IP databases.

Figure 17: Normalized number of active users inﬁber (top) and DSL (bottom): total ﬁber active
users, heavy-hitters and normal users90

hkd amr iwt myg akt ygt fks ibr tcg gnm stm chb tky kng ngt tym isk fki yns ngn gif szk aic mie sig kyt osk hyg nar wky ttr smn oky hrs ygc tks kgw ehm kch fko sag ngs kmt oit myz kgs okn
5.1 .49 .40 1.6 .45 .39 .94 1.4 .99 .82 5.0 4.8 11 7.5 1.0 .82 1.1 .40 .54 .85 1.3 2.3 6.0 1.2 .83 2.3 6.3 3.1 .63 .62 .28 .17 1.5 1.6 .51 .45 .51 .71 .52 2.3 .20 .61 .77 .46 .37 .30 .33
3.6 1.3 .44 2.3 .35 .55 1.2 1.5 1.0 .87 5.2 5.3 11 7.6 1.2 .55 1.0 .34 1.2 .77 1.3 2.2 5.6 .71 .66 2.2 6.6 3.2 .72 .61 .45 .11 1.3 1.5 .50 .27 .64 .77 .52 2.1 .23 .35 .88 .30 .30 .26 .34
4.0 .53 1.3 1.8 .48 .49 .96 1.8 1.3 .98 5.4 4.9 13 7.1 1.7 .72 .81 .44 .41 .65 2.0 2.1 6.4 1.1 .91 2.5 6.3 3.9 .91 .58 .30 .13 1.7 1.6 .54 .38 .74 .72 .46 2.4 .26 .49 .79 .46 .43 .54 .34
3.6 .86 .64 2.5 .50 .56 1.1 1.6 .93 .87 5.3 4.9 11 7.5 1.0 .63 .99 .36 .47 .78 1.3 2.1 5.8 .97 .60 2.4 6.5 3.1 .75 .54 .27 .10 1.3 1.6 .61 .37 .45 .82 .50 2.4 .20 .41 .77 .42 .40 .35 .35
3.5 .32 .37 1.6 .87 .35 .91 1.3 .89 .90 5.0 4.7 11 7.7 .94 .71 .98 .27 .62 .67 1.4 2.1 6.3 .84 .85 2.3 5.9 3.0 .68 .39 .25 .10 1.3 1.6 .51 .34 .55 .94 .41 2.2 .14 .21 .64 .51 .32 .33 .21
3.1 .43 .47 1.4 .63 .86 1.2 1.5 1.0 .67 5.1 4.8 12 6.4 .98 .71 1.1 .37 .54 .87 1.4 1.8 5.7 1.1 .68 2.3 6.5 3.6 .72 .40 .33 .11 1.3 1.4 .41 .38 .49 .66 .38 2.2 .20 .47 .63 .35 .38 .30 .30
3.5 .56 .41 1.5 .38 .55 2.2 1.8 .92 .91 4.8 4.1 10 8.7 1.0 .57 1.1 .50 .46 .66 1.4 2.1 5.8 1.2 .79 1.9 6.3 2.7 .50 .46 .27 .13 1.3 1.5 .65 .29 .51 .60 .45 5.3 .20 .47 .60 .33 .51 .29 .31
3.5 .40 .45 1.5 .28 .46 .99 2.0 .93 .85 4.8 4.2 11 6.7 1.1 .98 .91 .39 .45 .74 1.5 2.3 5.6 .89 .74 2.0 6.6 3.9 .58 .50 .30 .11 1.3 1.6 .63 .40 .53 2.1 .54 2.9 .22 .36 .74 .48 .34 .72 .40
4.1 .60 .52 1.5 .45 .43 1.2 1.7 1.8 .93 4.7 4.9 12 7.3 1.2 .64 1.3 .41 .43 .91 1.3 1.8 5.8 .93 .64 1.9 6.5 3.3 .74 .53 .25 .22 1.8 1.5 .47 .39 .50 .78 .41 3.1 .29 .41 .95 .39 .29 .34 .37
3.1 .46 .44 1.4 .43 .56 .00 1.4 1.3 1.7 5.2 4.8 12 7.3 1.1 .59 .90 .45 .46 .85 1.2 2.4 6.0 .92 .69 2.1 6.4 3.1 .65 .89 .35 .17 1.2 1.8 .52 .30 .61 .93 .49 2.5 .21 .35 .92 .42 .41 .40 .34
3.9 .48 .41 1.6 .44 .53 .88 1.5 .87 .85 5.9 5.0 12 7.4 1.1 .74 1.0 .49 .52 .91 1.3 2.3 5.9 .95 .78 2.0 6.5 3.2 .61 .52 .29 .12 1.5 1.6 .58 .35 .54 .81 .52 2.4 .25 .38 .64 .48 .42 .34 .34
3.6 .54 .42 1.7 .50 .48 .94 1.5 1.2 .97 5.2 5.4 12 7.8 1.2 .64 .97 .81 .48 .80 1.2 2.0 6.2 .92 .74 2.0 6.4 3.3 .75 .57 .33 .14 1.4 1.5 .46 .33 .50 .74 .45 2.4 .16 .33 .69 .46 .34 .38 .33
3.4 .44 .39 1.5 .43 .44 .92 1.4 .86 .80 5.2 5.0 14 7.3 1.1 .65 1.1 .35 .37 .98 1.3 2.0 5.5 .89 .71 1.8 6.2 3.0 .63 .51 .27 .14 1.2 1.9 .46 .32 .48 .67 .53 2.4 .20 .32 .68 .47 .31 .31 .31
3.8 .54 .45 1.4 .46 .46 1.0 1.5 .95 .83 5.2 4.8 12 8.5 1.0 .66 .99 .49 .42 .89 1.2 2.2 5.6 .00 .74 2.0 7.3 3.0 .64 .54 .29 .17 1.3 1.6 .49 .35 .50 .80 .45 2.8 .20 .31 .72 .45 .31 .32 .30
3.4 .51 .49 1.4 .42 .52 .97 1.3 1.1 .82 4.9 5.1 12 6.9 2.3 .74 .95 .45 .53 .71 1.2 2.2 6.3 1.1 .75 2.2 6.0 3.3 .52 .49 .24 .16 1.5 1.6 .44 .38 .57 .76 .54 2.3 .20 .43 .76 .86 .31 .34 .42
3.5 .46 .33 1.3 .43 .56 .91 1.3 .83 .73 4.9 4.2 11 7.1 1.1 4.1 1.5 .46 .58 1.0 1.3 2.3 6.2 .84 .69 2.1 7.2 3.2 .58 .81 .31 .32 1.3 1.4 .51 .67 .47 .72 .42 2.3 .23 .30 .78 .54 .35 .42 .31
4.2 .57 .41 1.5 .45 .39 .86 1.3 .90 .72 5.3 5.1 9.8 6.9 .99 .80 2.7 .62 .39 .81 1.3 2.7 5.9 1.2 .63 1.8 7.0 3.1 .56 .45 .31 .15 1.6 1.5 .50 .44 .43 .79 .38 2.3 .15 .45 .83 .46 .28 .35 .36
3.7 .38 .49 1.3 .38 .45 .85 1.7 .92 .74 5.3 4.8 10 7.2 .92 1.1 1.1 1.8 .36 .96 1.6 2.9 7.2 1.4 .62 2.1 8.8 3.2 .66 .50 .25 .11 1.3 1.8 .40 .66 .40 .66 .45 2.7 .20 .41 .79 .42 .27 .38 .28
4.3 .52 .37 1.8 .33 .50 1.4 1.6 .87 .76 5.8 4.9 11 6.5 .84 1.0 .89 .51 1.7 .81 1.3 2.3 5.9 .89 .81 1.9 6.9 3.4 .81 .62 .40 .20 1.5 1.6 .37 .48 .77 .75 .44 2.3 .15 .31 .76 .61 .41 .39 .35
3.7 .51 .57 1.7 .46 .49 .96 1.3 1.0 .70 4.9 4.8 11 8.0 1.1 .75 .89 .51 .45 3.2 1.3 2.2 6.3 1.1 .71 1.9 6.6 3.0 .82 .43 .29 .24 1.3 1.7 .51 .45 .57 .74 .47 2.5 .25 .36 .77 .47 .29 .42 .35
4.1 .60 .34 1.5 .42 .42 .99 1.5 .87 .88 5.1 4.2 10 7.0 1.3 .66 1.1 .51 .35 .99 2.2 2.6 6.0 1.2 .66 2.6 7.2 3.8 .71 .41 .29 .16 1.5 1.5 .58 .50 .99 1.0 .73 2.8 .17 .38 .73 .45 .39 .34 .55
3.5 .50 .43 1.4 .40 .45 .91 1.4 .94 .91 5.3 4.5 10 8.6 1.0 .78 1.2 .52 .42 .77 1.4 3.7 6.0 1.1 .77 2.5 6.0 3.2 .70 .84 .29 .14 1.4 1.6 .58 .35 .60 .70 .50 2.5 .26 .32 .74 .50 .47 .33 .40
3.6 .53 .38 1.4 .47 .46 .94 1.3 .84 .82 4.8 4.6 11 6.9 1.1 .67 .94 .58 .39 1.0 1.5 2.0 7.3 1.0 .69 2.3 6.5 3.2 .68 .62 .26 .16 1.4 2.5 .54 .40 .51 .84 .42 2.5 .29 .40 .72 .47 .38 .31 .31
5.4 .36 .31 1.6 .48 .39 1.2 1.3 .91 .77 6.2 5.0 10 7.3 1.1 .74 1.1 .79 .41 .95 1.4 2.3 6.3 1.7 .83 2.1 6.8 3.0 .62 .60 .36 .12 1.6 1.9 .44 .40 .49 .74 .35 2.4 .18 .36 1.1 .47 .36 .38 .26
3.3 .45 .45 1.4 .49 .35 1.0 1.5 1.1 .83 4.8 4.7 11 6.6 .99 .50 .88 .38 .47 .75 1.4 2.0 5.8 1.3 1.7 6.0 6.2 3.6 .69 .68 .29 .39 1.4 1.6 .50 .27 .47 .81 .51 2.4 .24 .27 .88 .27 .30 .31 .38
3.6 .49 .42 2.1 .41 .45 .88 1.3 .91 .89 4.9 4.3 10 7.4 1.0 .65 1.1 .31 .41 .71 1.4 2.3 5.6 1.0 1.1 3.5 6.4 4.2 .69 .56 .41 .15 1.4 1.6 .49 .38 .47 .67 .53 2.2 .22 .32 .67 .55 .36 .28 .31
3.3 .44 .37 1.3 .41 .46 .93 1.3 .87 .80 4.8 4.4 12 7.1 .87 .71 1.1 .45 .43 .85 1.5 1.9 5.7 1.0 .75 2.0 8.9 3.0 .74 .54 .37 .16 1.3 2.9 .51 .35 .57 .73 .66 2.7 .18 .34 .65 .42 .34 .31 .39
3.6 .50 .43 1.4 .45 .53 1.0 2.0 .83 .84 5.2 4.7 11 6.9 1.3 .69 .93 .44 .45 .80 1.4 2.2 6.4 1.0 .69 2.4 7.2 4.5 .61 .58 .28 .21 1.6 1.7 .53 .40 .51 .96 .58 2.4 .23 .43 .71 .45 .38 .33 .38
3.7 .63 .55 1.5 .51 .48 1.2 1.2 .92 .90 4.8 4.9 11 7.1 1.4 .58 1.2 .40 .44 1.1 1.1 2.6 6.2 .91 .95 2.0 7.9 3.0 1.2 .71 .27 .19 1.2 2.1 .45 .40 .63 .84 .60 2.5 .18 .40 .65 .46 .58 .36 .29
5.1 .40 .41 1.8 .38 .36 1.0 1.3 1.0 .88 4.6 4.4 12 7.7 .93 .75 1.1 .57 .57 .79 1.3 2.8 5.8 1.1 .62 2.8 7.8 3.0 .73 2.2 .49 .11 1.5 1.5 .47 .21 .48 .78 .60 2.6 .20 .36 .73 .33 .45 .46 .37
4.2 .33 .72 1.2 .42 .35 1.2 1.5 .86 .95 5.2 4.6 11 7.5 1.0 .79 1.6 .31 .63 .79 1.5 2.0 6.5 1.2 .69 2.6 7.2 3.4 .67 .60 .45 .07 1.4 1.7 .50 .29 .75 .84 .50 2.3 .39 .39 .80 .37 .47 .46 .48
5.3 .31 .53 2.1 .51 .28 .92 1.5 1.1 1.2 5.0 4.3 10 7.3 .92 .53 .71 .65 .51 .89 1.4 2.5 6.5 .74 .54 1.8 7.0 2.8 .54 .85 .46 2.5 2.4 1.8 .24 .92 .48 .55 .40 2.4 .10 .34 .59 .36 .28 .38 .28
3.6 .45 .28 1.3 .36 .34 .99 1.2 .79 .81 5.2 4.8 10 6.6 1.0 .59 .94 .62 .45 .72 1.3 1.9 6.1 1.1 .66 2.0 6.6 4.1 .62 .48 .28 .14 2.8 1.6 .51 .39 .56 .82 .68 2.5 .19 .33 1.2 .49 .34 .43 .32
4.1 .48 .47 1.5 .45 .54 .95 1.7 .97 .91 5.2 4.6 10 7.1 1.1 .81 1.8 .48 .40 .75 1.4 2.2 6.0 .99 .87 2.1 6.7 3.2 .64 .52 .37 .18 1.6 2.0 .44 .34 .61 .88 .41 2.8 .23 .36 1.7 .53 .31 .37 .31
4.0 .55 .48 1.7 .54 .44 1.2 1.4 .95 .95 5.4 4.8 10 7.7 1.1 .76 .89 .32 .39 .00 1.5 2.4 5.9 1.0 .66 2.2 6.2 2.9 .52 .57 .32 .09 1.6 1.7 .77 .36 .49 .76 .48 2.6 .16 .29 .84 .41 .28 .38 .34
5.4 .42 .25 1.7 .42 .39 1.0 1.3 .96 1.0 5.8 4.3 9.9 7.6 1.1 .94 1.3 .75 .41 .74 1.1 2.3 7.3 1.1 .74 2.2 6.5 3.5 .92 .52 .25 .23 2.4 1.5 .65 1.5 .72 1.4 1.1 2.4 .11 .24 1.5 .36 .27 .35 .31
6.6 .45 .36 1.6 .44 1.4 1.1 1.5 1.1 .98 7.1 4.4 10 6.8 .97 .72 .94 .41 .55 .86 1.3 2.0 5.8 1.0 .71 1.9 6.3 3.4 .91 .55 .28 .23 1.6 1.7 .81 .59 .74 .67 .74 2.8 .28 .36 .88 .38 .30 .37 .29
3.5 .53 .40 1.5 .43 .43 1.0 3.2 1.1 .95 4.7 4.6 11 6.4 1.0 .72 1.3 .32 .29 .74 2.1 1.8 8.7 .90 .65 1.7 5.8 3.0 .60 .86 .33 .12 1.2 2.0 .46 .32 .45 4.4 .41 2.5 .15 .30 .66 .34 .34 .34 .27
3.9 .50 .33 1.6 .39 .32 1.2 1.5 .82 .81 4.2 4.6 9.4 7.1 1.0 .58 .88 .53 .50 .80 1.8 2.3 5.9 .86 .76 2.0 8.2 4.4 .57 .72 .38 .22 1.7 1.4 .54 .32 .66 .71 4.6 2.6 .33 .35 .81 .40 .50 .36 .43
3.7 .46 .34 1.5 .38 .45 1.1 1.7 1.2 .96 5.3 4.7 12 7.4 1.0 .68 1.1 .72 .45 .77 1.4 2.4 6.3 1.0 .78 1.9 6.6 3.2 .67 .62 .27 .15 1.5 1.7 .48 .37 .54 .75 .51 4.2 .22 .33 .88 .41 .35 .33 .29
3.9 .49 .53 1.6 .34 .53 .96 1.1 .85 .87 6.0 5.3 10 8.1 1.4 .81 .91 .46 .33 .89 1.3 2.9 6.7 1.2 .66 1.9 6.5 3.2 .63 .44 .25 .25 1.9 1.6 .49 .42 .47 .72 .65 2.9 .72 .35 .89 .52 .43 .33 .58
4.5 .36 .40 1.4 .42 .41 .82 1.2 1.3 .80 5.2 5.0 10 6.8 1.2 .76 1.0 .47 .33 .50 1.3 2.1 5.9 .61 .68 2.5 8.7 3.1 .69 .52 .32 .16 1.5 1.4 .63 .30 .43 .72 .50 2.3 .20 1.1 .75 .30 .44 .31 .33
4.4 .61 .52 2.1 .40 .36 1.1 1.1 .96 1.1 4.6 5.1 10 6.8 .96 .63 1.2 .39 .46 .87 1.7 1.9 6.3 .86 .83 2.2 6.1 2.7 .64 .73 .38 .18 1.9 1.6 .72 .63 .60 .74 .57 2.9 .23 .40 2.4 .38 .58 .40 .29
4.3 .29 .25 1.6 .46 .49 .93 1.2 1.0 .77 4.7 5.1 10 8.3 1.6 1.3 1.0 .33 .57 .68 2.1 2.6 6.5 .85 .79 2.2 5.8 3.1 .79 .61 .41 .16 1.8 1.7 .84 .32 .48 .80 .37 3.5 .26 .34 1.1 .34 .45 .45 .36
4.1 .44 .35 1.6 .28 .29 1.1 1.4 .85 1.2 4.8 4.5 9.1 6.8 .80 .58 .92 .27 .50 1.0 1.4 2.3 5.6 1.4 .65 2.1 5.8 3.0 .76 .50 .47 .12 1.6 1.7 .45 .27 .60 .49 .73 2.6 .18 .49 .88 .41 .95 .39 .41
4.3 .50 .45 1.4 .55 .48 .89 2.0 .86 .94 5.6 5.2 11 7.9 1.2 .61 1.1 .52 .66 .99 1.6 2.3 5.9 .94 .86 1.8 7.5 3.5 .69 .58 .27 .11 1.7 1.7 .54 .38 .52 1.2 .47 2.8 .28 .26 .91 .53 .38 .55 .41
3.7 .47 .39 1.8 .38 .38 .91 1.3 1.1 1.0 5.9 4.7 10 6.4 1.0 .77 .84 .36 .42 .71 1.7 2.3 6.1 1.3 .64 1.9 6.8 3.6 .57 .48 .31 .11 1.4 1.4 .68 .38 .52 .91 .44 2.8 .39 .28 .79 .31 .23 .34 1.2
100
101
102

10
Cumulative distribution

hkd
amr
iwt
myg
akt
ygt
fks
ibr
tcg
gnm
stm
chb
tky
kng
ngt
tym
isk
fki
yns
ngn
gif
szk
aic
mie
sig
kyt
osk
hyg
nar
wky
ttr
smn
oky
hrs
ygc
tks
kgw
ehm
kch
fko
sag
ngs
kmt
oit
myz
kgs
okn

-1

10

10

-2

-3

10

User to user
User to domestic

-4

10 1

Figure 18: Traﬃc matrix of 47 prefectures normalized to the source prefecture. The columns have
similar values indicating that the distributions ofdestinations are similar among diﬀerent prefectures.

10100
Number of dominant peers

1000

Figure 19: CDF of dominant peer counts for userto-user and user-to-domestic
3

Number of dominant peers

10

All the rows in the matrix have similar distributions, thatis, users access similar destinations regardless of the user
location. It is also conﬁrmed that traﬃc volume is roughly
proportional to prefectural population. The traﬃc local tothe prefecture is on the diagonal line from the upper left tothe bottom right, and is only 2-3% of the total volume for all
the prefectures. On the other hand, we cannot identify any
increase in traﬃc to neighbor prefectures. A similar result
was found when the distribution is normalized to the destination prefecture. The results suggest that Internet traﬃchas very poor locality, in contrast to telephone communication where users tend to talk to nearby neighbors. However,this phenomenon might just be the behavior of dominant
applications rather than the fundamental nature of Internet
communication.
In order to distinguish application types in user-to-user
traﬃc, we investigated the number of peers for each user.
Before the experiment, we expected to observe two application types: a small number of peers for video-streaming anddownloading from servers, and a large number of peers forpeer-to-peer ﬁle-sharing.
To observe the number of peers by unique IP address, it isnecessary to exclude peers with small traﬃc volumes since
the tail of the distribution is long due to small transactions
such as DNS lookups and web browsing. Thus, each user’s
peers are sorted inversely by volume, and then, the numberof peers exceeding the 50th-percentile of the user’s traﬃcvolume is counted. We call this number, the dominant peer
count. Our data set consisted of traﬃc from one day taken
on July 5th, 2005. The inbound traﬃc for those users who
used more than 1GB were extracted for analysis. The destination peer types are classiﬁed by the geo-IP databases into
residential users, domestic and others. Then, each user isclassiﬁed by the largest destination type into the user-touser group, the user-to-domestic group and others.
Figure 19 shows the (complementary) cumulative distribution of dominant peer counts for user-to-user and userto-domestic for comparison. For the majority of the users
in the upper left region in the graph, the user-to-user group
has a much larger number of peers than the user-to-domestic
group; 80% of the user-to-user group at the horizontal line
have less than 18 dominant peers while 80% of the user-todomestic group have only less than 4.7 dominant peers. For
those who have many peers, the diﬀerence becomes smaller

2

10

1

10

0

10

-1

10

0

1

10
10
Daily Inbound Traffic [Gbps]

2

10Figure 20: Correlation between dominant peer
counts and traﬃc volumes in user-to-user traﬃcsince the domestic group also includes ﬁle-sharing users inorganizations connected by leased-lines.
Figure 20 shows the correlation between dominant peer
counts and the traﬃc volumes for the user-to-user group on
a log-log scale. There are a surprisingly wide range of peer
numbers regardless of the traﬃc volume. Some users communicate with more than 1000 peers but those outliers above
the graph range seem to perform scanning. A large numberof users communicate with only one or a few peers even in
the high-volume region but similarly many users communicate with 10-100 peers. If ﬁle-sharing applications have atypical transfer size for each peer, we should be able to observe positive correlation between peer numbers and traﬃcvolumes. Although a positive correlation coeﬃcient (0.28)
is observed, it spans a wide range of traﬃc volume, and theextreme heavy-hitters with a few peers do not follow the correlation. This suggests that high-volume traﬃc is generated
not only by peer-to-peer ﬁle-sharing but also by other applications such as content-downloading from a single server.
A plausible explanation for the large variance between peer
numbers and traﬃc volumes is that many users use bothﬁle-sharing and downloading at diﬀerent ratios.
On the other hand, the correlation plot for the user-todomestic group (omitted due to the space constraints) has aslight negative correlation coeﬃcient (-0.12). That is, heavyhitters in this group tend to communicate with a small number of peers when compared with the user-to-user group.
105. RELATED WORK

work is, to the best of our knowledge, the ﬁrst involving thecollection of long-term measurements from multiple ISPs toestimate nation-wide traﬃc volume, and the ﬁrst investigating user-to-user traﬃc in both ﬁber and DSL access lines.

The properties of residential broadband traﬃc diﬀer considerably from those of academic or oﬃce traﬃc often seen
in literature. The peak hours have shifted from oﬃce hours
to evening hours, and the constantly ﬂowing portion of daily
traﬃc ﬂuctuations is much larger than those found in earlier
reports [4, 5, 25].
There is little solid work in literature that tries to estimate
the growth rate of Internet traﬃc. Although old studies [23,3] exist, it has become harder and harder after the privatization of the Internet in mid 90s. Recently, Odlyzko analyzed
various aspects of traﬃc growth, and reported the growthrate of 100% per year for the U.S. in 2003 [19]. The IX trafﬁc in Figure 1 shows that the Internet traﬃc in Japan is still
steadily growing although the growth rate has slowed down
after 2002 and has been stable at about 50% per year for thelast few years. Statistics in governmental reports of Hong
Kong and Australia also show similar trends in their traﬃc
growth [9, 1]. Probably, it is partly because broadband deployment has reached most technically conscious users, andpartly because the IT industry is still recovering from thepost dot-com bubble economy.
Our results are consistent with earlier measurements ofpeer-to-peer traﬃc; peer-to-peer traﬃc is dominant in commercial backbones [24], highly variable and skewed among
participating nodes [26, 27, 8], and exhibits behavior considerably diﬀerent from traditional web traﬃc [7]. However,measurement techniques relying on known port numbers to
identify applications can no longer be applied since peer-topeer traﬃc is shifting from known to arbitrary ports [14].
Among various peer-to-peer measurements, a study ofFrance Telecom’s ADSL networks [24] is similar to our percustomer analysis in monitoring access lines and comparing traﬃc volumes among data sets over a year. However,their focus is on ﬁle-sharing applications and the monitoring
method relies on known port numbers. The results showing
two distinct user segments are considerably diﬀerent fromours, probably due to the ﬁber user ratio and diﬀerences in
popular application software from our measurements.
Many access technologies employ asymmetric line speed
for inbound and outbound based on the assumption that
content-downloading is dominant for average users. However, this assumption does not hold in our measurements.
Although many studies report the asymmetric nature ofpeer-to-peer traﬃc [27, 26, 24], it is clear from our comparison between ﬁber and DSL users that the bandwidth
demands of applications and users are not asymmetric, andthe deployment of symmetric access will change traﬃc patterns.
It is known that, in general, peer-to-peer traﬃc exhibits
very poor geographic locality [16, 15]. The intersection between query sets across diﬀerent regions is also very small[16] partly due to language and cultural barriers [10, 24].
However, modern peer-to-peer networks are characterized
by the small-world model (i.e., small diameter and highly
clustered topologies) [28], which could lead to heterogeneous
behavior in diﬀerent geographic regions [16]. Our result ofpoor locality in user-to-user traﬃc is consistent with others,though the granularity of the analysis is only at the prefectural level.
This paper focuses on user-to-user traﬃc rather than peerto-peer ﬁle-sharing. Our results show that ﬁle-sharing is not
the only dominant application in user-to-user traﬃc. Our

6.

IMPLICATIONS

Our per-customer measurements reveal the behavior ofresidential traﬃc in depth. At ﬁrst, we noticed a large skew
in traﬃc usage: the top 4% of heavy-hitters account for
75% and 60% of inbound and outbound traﬃc, respectively.
Fiber traﬃc accounts for 86% and 75% of inbound and outbound traﬃc, respectively. We tend to attribute the skews
to the divide between a handful of heavy-hitters and the restof the users. Our in-depth analysis, however, shows the existence of diverse and widespread heavy-hitters who appear
to be casual users rather than more dedicated users. In addition, the total traﬃc behavior seems to reﬂect the balanceof the diversity.
For example, the large skew in per-customer traﬃc seems
to be caused by a small number of heavy-hitters but, in fact,
the distribution of per-customer traﬃc is heavy-tailed and
it is diﬃcult to draw a line between heavy-hitters and therest of the users. The large skew in traﬃc volume between
ﬁber and DSL is not caused by qualitative diﬀerences in the
behaviors of ﬁber and DSL users but simply by the larger
percentage of heavy-hitters among ﬁber users. The domination of user-to-user traﬃc in residential traﬃc seemingly
points to peer-to-peer ﬁle-sharing but it is apparently a mixture of ﬁle-sharing and content-downloading. All the results
indicate that the perceived divides are actually caused bydiversity, or at least the divides are blurred by diversity. At
the same time, the overall user behavior reﬂects the balanceof this diversity, but it is sometimes dictated by the mostinﬂuential group.
We can no longer view heavy-hitters as exceptional extremes since there are too many of them, and they are statistically distributed over a wide traﬃc volume range. It is
more natural to think they are casual users who start playing
with new applications such as video-downloading and peerto-peer ﬁle-sharing, become heavy-hitters, and eventuallyshift from DSL to ﬁber. Other users subscribe to ﬁber ﬁrst,
and then, look for applications to use the abundant bandwidth. These casual users do not pay attention to underlying technologies and are capricious; their behavior would be
easily aﬀected by social, economical or political factors. The
implication is that, if a new attractive application emerges,
a drastic rise could occur in traﬃc usage. By the same
token, traﬃc could decrease. For example, current peerto-peer applications exhibit poor locality but locality-aware
applications would make better use of backbone bandwidth.
Current peer-to-peer ﬁle-sharing algorithms are designed to
ﬁll the narrow upstream bandwidth of DSL and are too aggressive for ﬁber access where it would not take too long to
download contents on demand.
In fact, the current total traﬃc volume is heavily impacted
by extreme heavy-hitters so that a slight change in the algorithms or charging policies could have a signiﬁcant impact
to backbone traﬃc. This situation can be regarded as atragedy of the commons [17] since the cost-eﬀective natureof the Internet architecture relies on statistical multiplexing
assuming bandwidth sharing among users. In fact, ISPs are
tempted to avoid congestion by suppressing traﬃc from extreme heavy-hitters, and it might be even desirable from a118.

view point of fair share of the infrastructural costs. At thesame time, it is just a stopgap since users as a whole areshifting towards high-volume usage.
As for the generality of our measurements, several aspects
are speciﬁc to Japanese traﬃc. One is the high penetrationof ﬁber access. It seems to take some time for other countries to deploy ﬁber access; even Korea that has the highest
broadband penetration ratio does not have widespread ﬁber
access [20]. Japan can be regarded as a model of widespread
symmetric residential broadband access. Another aspect isfairly closed domestic traﬃc. The current situation is partly
due to language and cultural barriers and partly due to richconnectivity within the country. The former could be common to other non-English speaking countries to some extent,
and the latter can be seen simply as the geographic concentration of bandwidth-rich users.

REFERENCES[1] Australian Bureau of Statistics: Internet Activity, Australia,
Mar 2005. http://www.abs.gov.au/.
[2] Cisco Sampled NetFlow.
http://www.cisco.com/univercd/cc/td/doc/product/software/
ios120/120newf%t/120limit/120s/120s11/12s sanf.htm.
[3] K. Claﬀy, G. Polyzos, and H. Braun. Traﬃc characteristics of
the T1 NSFNET backbone. In INFOCOM’93, pages 885–892,
San Francisco, CA, Mar. 1993.
[4] A. Feldmann, A. G. Greenberg, C. Lund, N. Reingold,
J. Rexford, and F. True. Deriving traﬃc demands for
operational IP networks: methodology and experience. In
SIGCOMM, pages 257–270, Stockholm, Sweden, Aug. 2000.
[5] C. Fraleigh, S. Moon, B. Lyles, C. Cotton, M. Khan, D. Moll,
R. Rockell, T. Seely, and C. Diot. Packet-Level Traﬃc
Measurements from the Sprint IP Backbone. IEEE Network,
17(6):6–16, Nov. 2003.
[6] K. Fukuda, K. Cho, and H. Esaki. The impact of residential
broadband traﬃc on Japanese ISP backbones. SIGCOMM
CCR, 35(1):15–21, Jan. 2005.
[7] K. P. Gummadi, R. J. Dunn, S. Saroiu, S. D. Gribble, H. M.
Levy, and J. Zahorjan. Measurement, modeling, and analysis ofa peer-to-peer ﬁle-sharing workload. In SOSP-19, pages
314–329, Bolton Landing, NY, Oct. 2003.
[8] L. Guo, S. Chen, Z. Xiao, E. Tan, X. Ding, and X. Zhang.
Measurements, analysis, and modeling of bittorrent-likesystems. In IMC2005, pages 35–48, Berkeley, CA, Oct. 2005.
[9] Hong Kong OFTA: Statistics of Internet Traﬃc Volume.
http://www.ofta.gov.hk/en/tele-lic/operator-licensees/
opr-isp/s2.html.
[10] M. Izal, G. Urvoy-Keller, E. W. Biersack, P. A. Felber, A. A.
Hamra, and L. Garceś-Erice. Dissecting bittorrent: Fivemonths in a torrent’s lifetime. In PAM2004 (LNCS3015),
pages 1–11, Antibes Juan-les-Pins, France, Apr. 2004.
[11] Japan Internet Exchange Co., Ltd. http://www.jpix.co.jp.
[12] Multifeed JPNAP service. http://www.jpnap.net.
[13] I. Kaneko. The Technology of Winny (In Japanese). ASCII,Tokyo, Japan, 2005.
[14] T. Karagiannis, A. Broido, N. Brownlee, kc claﬀy, andM. Faloutsos. Is p2p dying or just hiding? In Globecom 2004,
pages 1532–1538, Dallas, TX, Dec. 2004.
[15] T. Karagiannis, P. Rodriguez, and D. Papagiannaki. ShouldInternet service providers fear peer-assisted content
distribution? In IMC2005, pages 63–76, Berkeley, CA, Oct.
2005.
[16] A. Klemm, C. Lindemann, M. K. Vernon, and O. P. Waldhorst.
Characterizing the query behavior in peer-to-peer ﬁle sharing
systems. In IMC2004, pages 55–67, Sicily, Italy, Oct. 2004.
[17] L. Lessig. The future of ideas. Random House, 2001.
[18] NSPIXP. http://nspixp.wide.ad.jp.
[19] A. M. Odlyzko. Internet traﬃc growth: Sources andimplications. In Optical Transmission Systems andEquipment for WDM Networking II, pages 1–15, San Jose,CA, Jan. 2003.
[20] OECD Broadband Statistics, December 2005.
http://oecd.org/sti/ict/broadband.
[21] T. Oetiker. RRDtool: Round Robin Database Tool.
http://ee-staﬀ.ethz.ch/∼oetiker/webtools/rrdtool/.
[22] T. Oetiker. MRTG: The multi router traﬃc grapher. In
USENIX LISA, pages 141–147, Boston, MA, Dec. 1998.
[23] V. Paxson. Growth trends in wide-area TCP connections.
IEEE Network, 8(4):8–17, July 1994.
[24] L. Plissonneau, J.-L. Costeux, and P. Brown. Analysis ofpeer-to-peer traﬃc on ADSL. In PAM2005 (LNCS3431),
pages 69–82, Boston, MA, Mar. 2005.
[25] M. Roughan, A. Greenberg, C. Kalmanek, M. Rumsewicz,
J. Yates, and Y. Zhang. Experience in measuring Internet
backbone traﬃc variability: Models, metrics, measurements
and meaning. In International Teletraﬃc Congress 18, 2003.
[26] S. Saroiu, P. K. Gummadi, and S. D. Gribble. A measurement
study of peer-to-peer ﬁle sharing systems. In MMCN’02, pages
156–170, San Jose, CA, Jan. 2002.
[27] S. Sen and J. Wang. Analyzing peer-to-peer traﬃc across large
networks. In IMW, pp. 137–150, Marseille, France, Nov. 2002.
[28] D. Stutzbach, R. Rejaie, and S. Sen. Characterizing
unstructured overlay topologies in modern p2p ﬁle-sharesystems. In IMC2005, pages 49–62, Berkeley, CA, Oct. 2005.
[29] Statistical Outlook of the Internet in Japan (in Japanese). The
Ministry of Internal Aﬀairs and Communications, 2005.

7. CONCLUSION
The widespread deployment of residential broadband access has tremendous implications on our lives. Although
its eﬀects on the Internet infrastructure are diﬃcult to predict, it is essential for researchers and industry to prepare
to accommodate innovations brought by empowered end
users. Extensive eﬀort to establish protected data sharing
mechanisms with commercial Japanese Internet backbone
providers has allowed us to achieve an unprecedented empirical analysis of a signiﬁcant segment of the Japanese residential broadband traﬃc.
The growth of residential broadband traﬃc has alreadycontributed to a signiﬁcant increase in commercial backbone
traﬃc. In our study, residential broadband traﬃc accounts
for two thirds of the ISP backbone traﬃc and is increasing
at 37% per year, which will force signiﬁcant reevaluation of
the pricing and cost structures of the ISP industry.
We have further studied residential per-customer traﬃc in
one of the ISPs, and investigated diﬀerences between DSL
and ﬁber users, heavy-hitters and normal users, and in geographic traﬃc matrices. We found that a small segmentof users dictates the overall behavior; 4% of heavy-hittersaccount for 75% of the inbound volume. The ﬁber users
account for 86% of the inbound volume. About 63% of theresidential traﬃc volume is user-to-user traﬃc that exhibits
diverse behavior. The distribution of heavy-hitters is heavytailed without a clear boundary between heavy-hitters andthe rest of users.
For future work, we will continue collecting aggregatedtraﬃc logs from participating ISPs. We are also planning todo per-customer traﬃc analysis from other ISPs, and hope tocompare our results with measurements from non-Japanese
ISPs.

Acknowledgments
We are grateful to the following ISPs for their support andassistance with our data collection; IIJ, Japan Telecom, KOpticom, KDDI, NTT Communications, POWEREDCOM,
and SOFTBANK BB. We would like to thank the Ministryof Internal Aﬀairs and Communications of Japan for theirsupport in coordinating our study. We would also like tothank Atsushi Shionozaki of Sony CSL, Randy Bush of IIJ,
k claﬀy of CAIDA, and the anonymous reviewers for theirvaluable input.

12