Special Feature

Survey of the State of P2P File Sharing
Applications
Keita Ooi†, Satoshi Kamei, and Tatsuya Mori
Abstract
Recent improvements in Internet access have been accompanied by a dramatic spread of peer-to-peer(P2P) applications. P2P file sharing is affecting telecommunications carriers, copyright holders, andother areas of society. Since P2P applications do not employ servers to transfer files, it is difficult to gather traffic information on a large scale. This article presents i) techniques for measuring P2P file-sharing
application traffic by collecting packets using a modified application and ii) measurement results for aGnutella network. It also examines the current scope of P2P file sharing and the types of files that areshared.

1. What is P2P file sharing?
The origins of P2P file sharing go back to the Napster service, which has become synonymous with
P2P. A computer running a P2P file-sharing application participates in a P2P file-sharing network as a“peer” of other computers running the same application. A P2P file-sharing application has two key functions. The first searches for a certain file on the harddisk of a peer via the network, and the second transfers a file discovered by that search back to therequesting computer either directly or via anotherpeer.
2. Purpose of survey
The increasing popularity of P2P file-sharing applications has led to explosive growth in Internet traffic.
Some telecommunications carriers have alreadyresponded to this increase by stipulating in user
agreements that users who generate excessive
amounts of traffic may have their contracts cancelled.
Thus, there is a need to evaluate the effects of suchtraffic on telecommunications carriers and to predict
its future effects. To this end, it is essential to survey
actual traffic generated by P2P file sharing and to† NTT Information Sharing Platform Laboratories
Musashino-shi, 180-8585 Japan
E-mail: ooi.keita@lab.ntt.co.jp

24

understand the nature of file sharing itself.
3. Various P2P file-sharing applications
Napster required a central server for file searching,which means that the traffic generated by searching
had a pattern similar to that of ordinary Web traffic.
Most Napster-generated traffic, however, consisted offile transfers made directly between peer computers
without passing through the central server. This typeof traffic generates a pattern quite different from thatof Web traffic. While Napster the company discontinued its P2P service, compatible servers and clientsusing the same protocol as the Napster application
continue to be used elsewhere.
For example, the WinMX application, which features a compatible-client function, has found widespread use in Japan. Of all the P2P traffic that now
exists on Japanese networks, WinMX is believed togenerate the most. In addition, Gnutella, which cameafter Napster, employs an architecture that does not
require a central server. It achieves autonomous-distributed communications among peer computers forsearching as well as for file transfer. The absence of acentral server makes it difficult for a third party todeny or control services.
The above P2P file-sharing applications have been
followed by others, and at present, the one responsible for the largest P2P network in the world isKaZaA. These new applications, however, do not
NTT Technical Review

Special Feature

Peer computer running P2P software
P2P packet
Originating peer
(1) PING(5) Query(2) PONG(2) PING(5) Query(4) PONG
P2P network(5) QueryHit

IP peerFig. 1. P2P network.

support multilingual file names, and in Japan, none ofthem has become as popular as WinMX.
In Japan, there is also an application called Winnythat was created on the basis of Freenet, a P2P filesharing application having a high degree of anonymity. Winny can generate sudden increases in traffic,
and there are reports claming that it is generating
more traffic than WinMX at some measurement locations within carriers’ networks.

Fig. 2. Gnutella operation flow.
GnutellaNet

Controlpackets

Measurement
pointFig. 3. Gnutella measurement concept.

4. Measurement on the application layer
As shown in Fig. 1, peer computers running a P2P
application form a logical P2P network that overlays
the IP network on a layer independent of the physicalnetwork. It is therefore difficult to obtain information
like the scale of users and the state of file sharing only
from measurements made on the IP layer. In otherwords, measurements on the IP layer must be combined with those on the application layer to obtain anoverall picture. Furthermore, when there are applications whose protocol specifications have not beenreleased, it is even more difficult to determine usage
patterns by measurements on the network layer. Inaddition, the type of information that can be collected differs from one application to another, whichmeans that special measurement techniques must be
developed for each application. In the following sections, we describe a traffic measurement techniquefor a logical network taking the Gnutella file sharing
application as an example, and we present measurement results.
4.1 Measurement technique
To make measurements, we modified a servant toVol. 2 No. 5 May 2004

collect control packets as shown in Fig. 2. In Gnutella, a P2P network is constructed through theautonomous operation of peer computers (called“servants” in Gnutella). All control packets forchecking presence, searching, and maintaining thenetwork are exchanged on this network. There are
five types of control packets as described below.
• PING: A packet issued to discover connected servants.
• PONG: A packet issued by a servant in response toa PING packet. It includes an address and available
capacity.
• Query: A packet issued to request a search. It
includes a search character string. The number oftimes that this packet can be transferred from one
servant to another is specified beforehand.
• QueryHit: A packet issued in response to a Query
packet by a servant that possesses the file requested. It includes the file’s size and URL.
• PUSH: A packet issued to request the sending of afile when the sending side is behind a firewall.
These packets are sent and received as shown inFig. 3. Operations at each step of this process aredescribed below.
25

Special Feature

1. The originating servant sends a PING to otherservants.
2. Servants receiving a PING return a PONG andthen send a PING containing the identifier ofthe originating servant to other servants.
3. Servants decrement a number-of-transfers variable every time a PING passes through.
4. A PONG arrives at the servant where the PING
originated traveling in reverse order via the servants through which that PING passed.
5. In a search, the originating servant sends out aQuery, and the servant that has the file in question returns a QueryHit to the originating servant.
4.2 Measurement conditions
File transfers that are performed directly between
two servants are not targeted for measurement here. It
must therefore be kept in mind that the results presented below are limited to information about files
targeted for retrieval. In addition, by simultaneously
measuring traffic on the network layer formed byoperating servants using a traffic measurement techTable 1. Measurement results for select applications.
Type
Gnutella
WinMX

Unique IP
Unique files
addresses

Total filecapacity

Average filecapacity

30,0523,883,752

1.3 PB

330 MB350

165,933

25 TB

150 MBP: 1015

T: 1012100

Frequency (%)

101

nique for a logical network, we also collected information about the P2P logical network and measured
the scale of that network. This technique can beapplied to P2P applications in general.
4.3 Measurement results(1) Network scale
Table 1 shows the number of unique IP addresses,
number of unique files, total file capacity, and average file capacity for Gnutella and WinMX as data
reflecting network scale. The results shown for
Gnutella were obtained over a 68-hour period on aweekend in the first half of 2003. Those for WinMX
were obtained by sample measurements for comparison. About 30,000 IP addresses were collected forGnutella making it possible to estimate the network
size. Note that even if more IP addresses were to becollected by making measurements over a longer
time period, the number of IP addresses does not simply equate to the number of users. In other words, theeffective period of IP addresses must be taken into
account.
(2) File size
We compared the sizes of files shared on Gnutella,
WinMX, and the Web. The resulting file-size histograms are shown in Fig. 4, where both axes are log
scales. Gnutella files were considerably larger overallthan Web files. Figure 5 shows the distribution ofshared-file extensions searched for on the Gnutella
network. The mp3 audio-file extension was mostpopular followed by avi and mpg
moving-picture file extensions thatgenerally correspond to large files.
(3) Distribution of number of references
Figure 6 shows the distribution of
the number of references in Gnutella
on a log-log scale. The x-axis represents the number of file references(n) and the y-axis represents the0.1
avi, wmv, asf(4.99%)
mp3 (57.39%)

0.01

Others(31.21%)

0.001
<10 K

10 to
100K

100K
1 to
10 to
to 1M
10M
100M
File size (byte): Web

Fig. 4. File size distribution.

26

100M
to 1G>1G: Gnutellazip(0.78%)
jpg(0.67%)rar (0.57%)
mpg, mpeg (4.39%): WinMX

Fig. 5. Distribution of file extensions in
Gnutella search words.

NTT Technical Review

Special Feature

Number of files referenced n times

10,0001000

100101110100
1000
Number of file references (n)10,000Fig. 6. Distribution of number of shared files in Gnutella.

number of files referenced n times. The linearity
exhibited by the plots in this figure is called Lotoka’slaw (or Zipf’s second law). In general, when recording what individual users select, the resulting shape(Lotoka-type distribution) conforms to this law. For alog-data Lotoka-type distribution, it is known, forexample, that if the total number of files is small compared with the total number of references, then linearity will not be maintained in the area corresponding to a small number of file references (upper leftarea). That is to say, the plot will form a curve in thedownward direction [1].
The log-data Lotoka-type distribution also indicates that cooperative filtering systems are effective[2]. And this suggests the possibility of applying P2P
file-sharing log to fields such as market analysis andmarketing.

identify P2P traffic is to store the features of each current P2P application in a pattern file (as in Ellacoyaand P-Cube). The drawback of these systems, however, is that new pattern files must be prepared whenever new or upgraded P2P applications appear, which
increases the possibility of erroneous results. The
need to monitor the contents (payload) of transmissions also makes it difficult to deal with large-scalesystems.
At NTT Service Integration Laboratories,
researchers are developing traffic separation systems
for identifying P2P traffic using the logical-network
traffic measurement technology introduced in thisarticle and information on peer activity collected bythat technology [3]. These systems will make it easier to observe the effects of P2P traffic on the networkand to perform independent control on separate typesof traffic.

5. Conclusion
References
At NTT Information Sharing Platform Laboratories, researchers are working on analysis systems thatcan utilize the historical properties in Lotoka distributions (as mentioned in this article) and ones that cananalyze and use histories on an even larger scale.
There is also a need for ongoing surveys to evaluate
and predict the various effects of user scale, whichtends to increase in a P2P file-sharing application.
The main approach adopted by systems that aim toVol. 2 No. 5 May 2004

[1]

[2]

[3]

K. Muranaka, M. Matsuda, M. Aida, T. Motohashi, and M. Sato,“Analysis of Internet Access Patterns in Finite Address Space,”
IEICE Technical Report, IN2001-56, 2001 (in Japanese).
T. Motohashi, M. Sato, and A. Kanai, “Advertising and MarketingTechnology for Portal Services,” NTT Technical Journal, Vol. 14,
No. 1, pp. 85-87, 2002 (in Japanese).
T. Mori, S. Kamei, and K. Ooi, “Measurement Analysis and Characteristics Evaluation of P2P Traffic,” IEICE Society Conference, SB3-1, Niigata, Japan, Sep. 2003 (in Japanese).

27Special Feature

Keita OoiTatsuya Mori

NTT Information Sharing Platform Laboratories.
He received the B.S. and M.S. degrees in mathematical science from Kyoto University, Kyoto
in 1996 and 1999, respectively. In 1999, hejoined NTT Information Sharing Platform Laboratories, Tokyo, Japan. His current interest is collaborative filtering.

NTT Service Integration Laboratories.
He received the B.S. and M.S. degrees inapplied physics from Waseda University, Tokyoin 1997 and 1999, respectively. Since joining
NTT in 1999, he has been researching computer
network measurement and analysis.

Satoshi Kamei
NTT Service Integration Laboratories.
He received the B.S. and M.S. degrees inapplied mathematics and physics from Kyoto
University, Kyoto in 1997 and 1999, respectively. In 1999, he joined NTT Service Integration
Laboratories. His current interests are measuring
and analyzing overlay network traffic.

28

NTT Technical Review