Letters

IPDM—A Novel Technology for High-speed
Downloading of Large-scale Contents
Hiroshi Yamamoto, Yoshinori Kawamura,and Isami Nakajima
Abstract
There are growing demands for the delivery of large-scale content over the network, such as video andmusic. To expand the delivery business, it is essential to shorten the time required for delivery. It has
become clear that using broadband networks alone is not a solution because the content to be downloaded is increasing enormously in size. This article introduces a new technology to solve this problem.

1. Expansion of content delivery servicesAs the use of always-on Internet connections andbroadband access becomes widespread, the businessof delivering music, video, and other types of content
is beginning to flourish. Since most content providers
assume customers are using ADSL (asymmetric digital subscriber line) access (at a few megabits per second), the main items of content delivered are compressed music files or short video clips. However, as
optical access spreads and the performance of thedelivery servers improves, demands are expected torise for the delivery of large-scale content, such as
high-quality PCM-encoded music, DVD videos witha duration of 1 to 2 hours, and games software (PCM:pulse code modulation). The content business market
is forecast to grow to 2.8 trillion yen by 2007 [1].
2. Problems
One key to expanding the content delivery businessand increasing customer satisfaction is a shorter content delivery time. If it takes only a few seconds or
minutes to deliver content, then depending on thecontent type, the on-demand delivery business willgrow rapidly [2].
To shorten the delivery time, we must increase the† NTT Service Integration Laboratories
Musashino-shi, 180-8585 Japan
E-mail: ipdm@lab.ntt.co.jp

Vol. 2 No. 8 Aug. 2004

network access bandwidth and raise the performanceof the delivery server. However, these measures alone
cannot achieve the speed needed for the delivery oflarge-scale content because, as content grows in size,delivery speed becomes constrained by two factors:
the throughput limit imposed by the network transfer
protocol and the limit on the transfer speed of thedelivery servers. These constraints are explained
below.
2.1 Limit on TCP throughput
The network transfer protocol commonly used forcontent delivery is TCP (transmission control protocol), which includes retransmission control for packets that are lost. The maximum throughput that can beachieved by TCP depends on the packet loss rate andthe round-trip delay between end terminals. Figure 1
shows how the theoretical maximum throughput of
one TCP session depends on the round-trip delay andpacket loss rate. These days, the round-trip delay isincreasing because of longer propagation distances
and the introduction of firewalls and load sharing
devices. It is difficult to reduce the delay caused bythese constraints, even if a highly broadband network
is introduced. A round-trip delay of 50 ms or more is
not uncommon, even for communication between
points within the same prefecture in Japan. (The maximum theoretical throughput is 16 Mbit/s with thewidely used TCP New Reno [3].) Therefore, even if
high-speed access (e.g., optical access) is used, TCP
still imposes a limit on the data throughput.
93

Letters

Round-trip delay time (ms)Theoretical TCP throughput calculated from packet loss rate and delay time(Assumptions: TCP New Reno, delayed ACK=2, packet retransmission interval=500 ms)
100
For reference:round-trip delay
between NTT R&D
Center in Tokyo and 80
the following sites
Site D (900 km)

Below 1 Mbit/s

60

1 to 2 Mbit/s

Site C (600 km)*Site B (1200 km)
Site A (50 km)

2 to 4 Mbit/s

40

4 to 8 Mbit/s
20
8 to 32 Mbit/s

From NTT R&D
Center in Tokyo
0Above 32 Mbit/s
0.2

0.4
0.6
Packet loss rate (%)

0.8

1* The round-trip delay is determined by not only the transmission distance but also equipment
implemented in the network. For site B, it seems that the influence of such equipment is quite
limited.

Fig. 1. Theoretical throughput of TCP New Reno.

Data transfer processing within a serverContentRs

Page cacheTemporarystorage
BufferStorage

Rc
NetworkRs

Server transfer rate

MemoryBottleneck

Sequential access: fast
Fast in any access method(several hundred Mbit/s)(several Gbit/s)
Random access: slow
(a few Mbit/s to several tens of Mbit/s)

Normally: Rs + Rc ≅ Rs
(several Mbit/s to several tens of Mbit/s)
When cache is available: Rc(several Gbit/s)

Fig. 2. Model for data transfer processing within a server.

2.2 Limit on the transfer rate of servers
The data transfer mechanism inside a delivery server can be modeled as shown in Fig. 2. Since network
interface cards (NICs) have become faster recently,
they are no longer a major limiting factor on the server transfer speed. Server transfer speed can be
expressed as the sum of the storage reading rate Rs
and the cache reading rate Rc.
Rs is high for continuous readout, but tends to beextremely low for random access. Access to a large
item of content by many users becomes random, so ahigh data transfer cannot be expected. Rc depends onthe cache hit rate. For a large item of content, thecache hit rate is necessarily low (Fig. 3).
94

Since neither Rs nor Rc can be increased, the server transfer speed often becomes the bottleneck forhigh-speed delivery.
3. IPDM: technology to overcome these limits
3.1 Basic idea
We have developed a high-speed data delivery technology called IPDM (intelligent parallel download
method), which overcomes the two limits that appear
as the content size increases. It can, for example,
deliver the contents of a DVD within a few minutes
over a 100-Mbit/s access line.
IPDM maximizes the use of the data available in theNTT Technical Review

Letters

For small-scale content
Small-scale content

Whole content iscached.

Small-scale content
Cache

Head

EndHeadEnd

Any data in the content
will make a cache hit.

For large-scale content
Large-scale content
Head

End
Only part of thecontent is cached.

Cache
Head

Data included in this area
causes a cache miss.

Cache hitCache missNon-cache area

EndCache areaFig. 3. Relationship between content size and cache hit rate.

IPDM download

Downloading using existing multiple TCP sessions
Head

Large-scale content

HeadEndHead

Six TCP sessions downloadfrom three servers

Clients read datafrom storage.
HeadEndLarge-scale content

Large-scale content

EndSix TCP sessions
download from threeservers
Large-scale content
End

Clients read datafrom cache.
Paralleldownload
control(IPDM)

Server A
Server B
Server C
With existing multiple TCP sessions, the larger the numberof sessions, the more sessions that read data directly fromthe storage.

Though multiple TCP sessions are created, the number ofsessions that read data directly from the storage will notincrease thanks to the readout point being moved to theoptimum cache point.

TCP’s temporal distance problem can be solved, but
the transfer rate may not be improved because theserver transfer performance will be degraded.
Readout point
Non-cache area
Cache area
(read from memory)

The technology can solve TCP’s temporal
distance problem without deteriorating the servertransfer performance.
Range of download
Readout point
assigned to each server(read from storage)

Fig. 4. Use of multiple TCP sessions with IPDM.

page cache within the server to overcome the limit on
the server’s data transfer rate. Specifically, it estimates which part of the content is in the cache andwhen a delivery request arrives from a client, IPDM
moves the readout point to the cache. This increases
the data transfer rate.
To overcome the limit of the TCP throughput,
IPDM uses multiple TCP sessions simultaneously.
Figure 4 shows a case in which a large item of conVol. 2 No. 8 Aug. 2004

tent is delivered in parallel from three servers (A, B,and C) to a single client using multiple sessions. Eachtriangle indicates the point where a TCP session isreading the content at a certain point in time. If theexisting method of downloading data using multiple
TCP sessions is used, the number of sessions reading
data from the storage device increases, which reduces
the server transfer rate (left side of Fig. 4). IPDM uses
its data availability estimation to deliver data from95

Letters

several caches. This maximizes the use of cacheddata, so it does not aggravate the server transfer rate
problem that normally arises when multiple sessions
are used (right side of Fig. 4).
3.2 Example of cache control
Figure 5 shows how the readout point is controlled
when delivery of a large-scale content to a new clientis started. Since the LRU*1 algorithm is used in theoperating system of most delivery servers, the data
most recently read by the client remains in the cacheat the point shown by s in Fig. 5. Consider the casewhere content delivery to the client starts at the headof the content (upper part in Fig. 5). During Times 1
to 3, data reading by Client C1 causes some of thedownloaded content to be stored in the cache. AtTime 4, when C2 begins to receive the content, thefront part of the previous content is lost in the cache.
With the conventional technology, which always
downloads from the head of the content, even C2
receives content from the storage device. As time progresses, C3 starts to read data from the storage
device. Thus, the three clients must share the readout
capability of the storage device.

Passage of time

*1 LRU (least recently used): a cache updating algorithm in which
the least recently used data is deleted first when new data is to becached.

In contrast, in IPDM at Time 4 when C2 begins toread data, it does not read the data in the storage
device but the data in the cache stored there for C1(lower part of Fig. 5). Similarly, C3 begins to readdata cached for C1 and C2. As a result, even though
all three clients are receiving data, only C1 is reading
data in the storage device. Since C1 monopolizes thestorage device, it can read data at high speed. C2 andC3 read cached data, which increases the data readout
speed.
3.3 IPDM functions
The functional arrangement of IPDM is shown inFig. 6. To use IPDM, it is necessary to install cachemanagement server software (Function 1) in the server and an IPDM plug-in (Function 2) in the user’sbrowser. It is not necessary to modify existing delivery servers at all. Therefore, a provider with an existing delivery server can adopt IPDM at a very low
cost.
3.4 Client interface
IPDM’s client interface is shown in Fig. 7. An
embedded Javascript program ensures that delivery isautomatically based on IPDM, so the user need not be
conscious of using IPDM.

Using the existing technologies (browser download tool)Direction of movement of readout points
HeadEndLarge content
I/O rate deteriorates
Time 1
greatly as the number ofC1clients increases.
Time 2C1Time 3Time 4
Time 5Time 6Time 7

Page cache isdivided.

C1
C2

C1

C2

C1

C3 C2

C1

C3

C1

C2

Limit of I/O rateis dominant.

Three clients areaccessing the samestorage at a time.

Passage of time

Using IPDM
Using IPDM
Direction of movement of readout points
HeadEndLarge content
I/O rate is constant even
Time 1
if the number of clients
C1
increases.
Time 2

Non-cache area
Cache area

C1

Time 3Time 4
Time 5Time 6Time 7

C2C1
C2C3

Readout point(read from cache)

Readout point is moved
to the optimum cachepoint.

C1C1
C2

Only one client isaccessing the storage.

C1

Limit of I/O rate
is resolved.

C3 C2 C1

Readout point(read from storage)
Clients: C1, C2, C3
I/O: input/output

Fig. 5. Cache control in IPDM.

96

NTT Technical Review

Letters

Technical point: Making the best use of the data stored in the page cache of the delivery server
Function 1: Optimum page cache navigation (management server)
Function 2: Download in accordance with Function 1 and reassemble files (software must be distributed to clients)
When the client downloads Content X
Delivery server(existing servers can be used as they are)Function 2: client(browser needs PDM plug-in)

X1

Procedure 3:
download as instructed

Reassembly

X1

X2

X2

X3

X1

Content X
X2
X3

Server 1

X1

Content X
X2
X3

Server 2

X1

Content X
X2
X3

Server 3

X3
Procedure 2:
instruction to download- Part X1 from Server 1,- Part X2 from Server 2,- Part X3 from Server 3(This is called “download list”)

Procedure 1: the client asks
to download Content X.

Stored inpage cache
Function 1: cache management server (newly installed)- Page cache information is collected.
Part X1 of Content X is cached in Server 1.
Part X2 of Content X is cached in Server 2.
Part X3 of Content X is cached in Server 3.
- The download method that makes the best useof page cache is derived.

Fig. 6. Functional layout of IPDM.

The client can readily use IPDM by simply installing an IPDM plug-in.
Downloading in the IPDM mode becomes possible by adding the followingJavascript tags to the content link. Users download using IPDM without being
conscious of its procedures.

<a href=‘javascript:IPDM.GetObject(“Content name”)’>Download</A><BR>

UsingIPDM
Usingbrowser<a href=“http://Server name/Content name”>Download</a>Fig. 7. Client interface of IPDM.

3.5 Throughput comparison with alternativetechnologies
We compared the throughput of IPDM and competing technologies: load balancer and Flashget (amethod of downloading data using multiple existing
TCP sessions) [4]. The results are shown in Fig. 8.
The left side of Fig. 8 plots the total throughput of six
servers for different volumes of data delivered. As thedata volume increased, the throughputs of the loadbalancer and Flashget decreased dramatically while
the throughput of the IPDM remained nearly constantat a high level.
The right side of Fig. 8 shows the throughput perclient for different values of round-trip delay. Withload balancer, as the round-trip delay increased, theproblem of TCP (temporal distance) became pronounced and throughput suffered. Although FlashgetVol. 2 No. 8 Aug. 2004

used multiple TCP sessions, the server transfer problem was prominent and kept the throughput almost
constant at a very low level. In contrast, when IPDM
was used, multiple TCP sessions could be used without any server transfer problem. Thus, it was possible
to download data at the maximum speed possible fora given bandwidth.
4. Future issues
We have developed technology for delivering dataat almost the maximum throughput of the broadband
network or delivery server used. We will study ways
to increase the reliability of this technology to guarantee operations in the event of data loss and we willexplore business opportunities.

97Letters

Effects of IPDM(Mbit/s)
700

Little effect on performance
if the volume of delivered
data increases.

600
Upper limit ofserver NIC
IPDM

500
400
300

Load balancer
Flashget20010004812162024 Existing method of
Total amount of data delivered (GB) downloading multiple
TCP sessions makes
Accessed simultaneously by 30 clients. the I/O problem
pronounced.
There were differences in content
popularity. No network emulation
applied.

Average throughput per client

Total throughput of six servers

Solves the server transfer
rate problem(Mbit/s)
100

Solves TCP’s temporal
Download is possible at thedistance problem
maximum speed available
for a given bandwidth even
if round-trip delay time
becomes larger.
Bandwidthavailable to users

8060

IPDM

40

Load balancer
Flashget20

Throughput is low
with load balancer
30 45 60 75 90 (ms)because of theRound-trip delay time
TCP problem.
NTT R&D Site A Site B Site C Site D
Center (50 km) (1200 km) (600 km) (900 km)
015Four 2-hour DVD programs were delivered toten clients. There were differences in content
popularity.

Fig. 8. Performance comparison between IPDM and competing technologies.

References
[1]
[2]
[3]
[4]

98

White Paper on Digital Content, Digital Content Association ofJapan, 2003 (in Japanese).
Nikkei Communication, Oct. 2, 2000 (in Japanese).
RFC2582: The NewReno Modification to TCP’s Fast Recovery Algorithm.
http://www.amazesoft.com/Yoshinori Kawamura
Senior Research Engineer, NTT Service Integration Laboratories.
He received the M.S. degree in mathematics
from Nagoya University, Nagoya, Aichi in 1989
and joined NTT the same year. He has beenworking in R&D of quality control for broadband
network services.

Hiroshi Yamamoto

Isami Nakajima

Research Engineer, Communication Traffic &Service Quality Project, NTT Service Integration
Laboratories.
He received the M.S. degree in informationand computer science from Waseda University,
Tokyo in 2001 and joined NTT Service Integration Laboratories the same year. He has beenworking on the architecture and performance
evaluation of content delivery networks. He has
also been contributing to ITU-T SG13 activities.

Group Leader, NTT Service Integration Laboratories.
He received the M.S. degree in mathematics
from Tohoku University, Sendai, Miyagi in1984. Since joining NTT Laboratories in 1984,
he has been engaged in research on network
design, traffic control, and QoS management. He
is a member of the Institute of Electronics, Information and Communication Engineers of Japanand the Operations Research Society of Japan.

NTT Technical Review