Selected Papers

QoS Management System for Video
Distribution Services
Akihiko Tsuno†, Yoshihiro Otsuka, and Sohei Majima
Abstract
We describe a quality-of-service (QoS) management system for the providers of video distribution services over IP (Internet protocol) networks. It detects degradation in user-perceived QoS by active measurement and locates the cause and it evaluates the QoS perceived by each user by passive measurement.
The system consists of active agents, passive agents, traffic generators, HMI (human-machine interface)terminals, management servers, and quality report servers. In this paper, we explain how to detect degradation by active measurement and locate its cause. We also describe an example of QoS evaluation by
active and passive measurement.

1. Introduction
Recent developments in the areas of informationand communication technologies have been truly
remarkable, and the Internet in particular has become
almost indispensable to most of us in our lives. The
growth of the Internet has certainly hastened the proliferation of broadband access lines and the emergence of an always-on connection environment. Capitalizing on these developments, a growing number ofproviders have entered the market to deliver video
over IP (Internet protocol) networks, with the resultthat video distribution services have become fairly
commonplace. However, we have also seen that video
distribution over broadband is vulnerable to qualityof-service (QoS) degradation, so service providers
need to manage the quality of the elements of the content distribution network, which consists of a distribution server and network equipment. To date, Internet
QoS has been managed by analyzing and responding
to device alarms. However, response-based management makes it difficult for providers to offer a video
service that will satisfy most customers because
device alarms of the server and the network manage
the QoS of the network layer. Moreover, the providers
measure delay time and packet loss as network-layer

QoS for the customer’s service level agreement(SLA). However, the measurement is not based on thecustomer’s viewpoint of the user-perceived QoS andis performed between only a small number points.
Therefore, it is important to check the QoS from thecustomer’s viewpoint and determine the locations of
distribution-QoS bottlenecks. What the customer
wants is to have the overall QoS managed from thevideo source through the network to his/her terminal.
In addition, it is important to respond promptly to auser’s inquiry about quality degradation. There are a
lot of management objects because we assume alarge-scale network. Therefore, QoS management isessential in order to detect and deal with QoS-related
issues.
In this paper, we describe a QoS management system from the viewpoint of the user-perceived QoS
and distribution-QoS bottleneck localization. It
detects degradation in user-perceived QoS by active
measurement and locates the cause and it evaluates
the QoS perceived by each user by passive measurement. This system reduces operations by the maintenance staff by regularly measuring the QoS and localizing degraded-QoS sections and equipment in the
network. Moreover, the providers manage the QoS
and can contribute to efforts to improve user satisfaction.

† NTT Network Service Systems Laboratories
Musashino-shi, 180-8585 Japan
E-mail: tsuno.akihiko@lab.ntt.co.jp

Vol. 4 No. 2 Feb. 200625

Selected Papers

2. QoS management requirements for video
distribution services
Since services provided over the Internet generally
traverse equipment belonging to multiple providers,the QoS of each participating provider (e.g., the QoSof their servers and networks) must be determined
and the user must be told what is the responsibility of
one provider and what is the responsibility of others.
This involves localizing and determining the causesof problems affecting QoS. If the problem resides ina provider’s own equipment, the provider must beable to identify the bottleneck and overcome it. The
required QoS management must be able to detect thedeterioration of transmission QoS over IP networks,
pinpoint where in the network and/or the iDC (Internet data center) the QoS degradation is occurring, andpinpoint the network equipment causing the problem
if the QoS problem lies in a network and/or iDC
under the provider’s own management.
i) Detecting the deterioration of distribution QoS
over IP networks
When the QoS of an IP network is measured, thestandard of deterioration varies depending on the typeof content and type of distribution application, so it isnecessary to provide different kinds of probe packets
that simulate each type of content and each type of
distribution application. For example, packet length,transmission interval, transmission time, and otherparameters can be set as generation conditions. Moreover, it is necessary to set the corresponding thresholds.
ii) Pinpointing where in the network and/or iDC the
QoS degradation is occurring
Measuring equipment is deployed at nodes between
networks, and bottlenecks are pinpointed by gradually narrowing down the search range by breaking up
the networks into ever smaller sections. The criteria
are based on the dominant factors affecting the users’
perceived QoS. Gathering QoS-related data from all
equipment across the whole network is impractical as
UserInterconnected
networksa means of pinpointing the bottleneck once degradation has been detected, simply because there is toomuch of it.
iii) Pinpointing the equipment causing the problem if
the QoS problem lies in a network and/or iDC
under the provider’s own management
The QoS problem is caused by a shortage ofresources. The equipment causing the problem may
include various types. The evaluation criteria couldbe, for example, the packet loss rate, line activity,
CPU (central processing unit) activity, and memory
capacity. The QoS-related data is collected from theequipment in a network and/or iDC. However, aprovider cannot collect QoS-related data from networks belonging to other providers.
3. Targeted network configuration and QoS
measurement
3.1 Content distribution network configuration
A content distribution network consists of an iDC,
the provider’s own network, and interconnected networks managed by other companies (Fig. 1). Therefore, an end-user’s PC is connected to an iDC via asequence of networks; one or more of them are managed by the provider, but the others are operated byother companies. The iDC includes one or more content servers, a load balancer, a firewall, and various
other pieces of equipment. While all networks use
many routers, a provider can only obtain QoS datafrom the routers on networks that it controls itself.
This QoS data is collected in a management information base (MIB), which can calculate packet loss
ratios and line activity ratios.
3.2 QoS measurement
3.2.1 Distribution QoS and user-perceived QoS
To efficiently and accurately grasp the users’ perceived QoS, it is necessary to clarify the relationshipbetween the perceived QoS and the distribution QoS
and identify the main factors causing QoS to decline.

Provider’s own
network

iDC
Content
serverClient PC

Content
serverFirewallLoadbalancer

Fig. 1. Content distribution network.

26

NTT Technical Review

Selected Papers

Distribution QoS can be broadly divided into server
QoS and network QoS. Since the primary factors contributing to reduced server QoS are CPU overload
and insufficient memory, we use CPU activity ratio
and remaining memory capacity as QoS criteria. And
considering that the primary causes of reduced IP network QoS are router or link overload, we use the IP
packet loss ratio as the QoS criterion in this case.
3.2.2 Measurement of IP network QoS andserver QoS
When the QoS of a server is measured, logs thatexist in the server are collected, and deterioration isdetected. The relationship between the log and theimage quality of each application must be examinedbecause the log is different in each application andthe threshold must be set.

Traffic generator

3.2.3 Definition of packet loss ratio
The packet loss ratio can be measured by an active
agent, as shown in Fig. 2. We use Pr (defined by Eq.
(1)) as a measure of the packet loss ratio. It is necessary to synchronize the time between the traffic generator and the active agent. Note however that if thelast packet fails to reach the active agent, this impedes
our ability to perform periodic measurements. We
deal with this potential problem by assigning anappropriate time-out value to the last packet. If thefirst packet fails to reach the active agent, we can recognize the loss of the packet by counting the numberof packets that reach it in the measurement time (T1– T0 + time-out value).
Pr = (Ns – Np)/Ns,(1)

where Ns is the number of packets sent by the traffic
generator between T0 and T1, Np is the number ofpackets received by the active agent that were sent by
the traffic generator between T0 and T1, and Xi is thenumber of lost packets.

Active agent

T0

4. QoS management system
X1

In a large-scale network, it is necessary to set up a
lot of active/passive agents. This large number of
agents enables us to localize the degraded-QoS section efficiently. Considering the effort involved, it is
not realistic for us to manually set the agents, synchronize them, operate them, and collect the results.
In addition, manual control would include the possibility of human error. Therefore, it is preferable toperform the setting and operation automatically froma remote site.
Our QoS management system is schematicallyshown in Fig. 3. QoS degradation is detected by peri-

Ns
X2
Np
T1

Fig. 2. Definition of packet loss ratio.
User

Interconnected
networks

Provider’s own
network

Client PC

iDC

Content
serverMIB

Content
serverFirewallPassive
agent

Activeagent

RouterTraffic
generator

Passive
agent

Passive
agent

Quality report Management
servers (Web)servers

Loadbalancer

HMI
terminals

Fig. 3. QoS management system for video distribution services.

Vol. 4 No. 2 Feb. 200627

Selected Papers

Table 1. Quality reports.
Purpose
ProvidersUsersi) Specifying the
distribution route· Confirmation of service
conditions· Selection of video content
delivery service

iDCs· Business for users· Network managementii) Localizing the degradedQoS section· Network managementiii) Localizing the degraded-QoS
equipment· Network management· Upgrading of equipment

odically measuring the QoS management criteria
near the users. The system consists of active agents,
passive agents, traffic generators, HMI (humanmachine interface) terminals, management servers,
and quality report servers. They are implemented insoftware.
To measure the QoS, the traffic generator sends
probe packets simulating actual content traffic overthe network. These are detected by the active agent,
which measures the packet loss ratio. The QoS of the
network as a whole is inferred from the data collected by the active agent and is used in the provisioningof network resources. The passive agents are
deployed in end-user PCs, in network equipment, andin the iDC and they support the customer service staff
by judging the QoS perceived by the user. Based on
various sources of information, user quality reports
are generated by quality report servers and then provided to users, providers, and iDCs (Table 1).
5. Localizing distribution-QoS bottlenecks
In the provisioning of network resources, it isimportant to detect the occurrence of QoS degradation and localize the bottleneck as quickly as possible. Our system can estimate the QoS of the network
as a whole between the traffic generator and active
agents by using active measurement to detect degradation of transmission QoS. If degradation is detected, the bottleneck is localized through the following
three steps:• Specifying the distribution route• Localizing the degraded-QoS section• Localizing the degraded-QoS equipment
After the bottleneck has been localized, the provider
can allocate additional bandwidth and resources,
upgrade the equipment, or take some other appropriate action.

5.1 Specifying the distribution route
When QoS degradation occurs, it is essential toidentify the route over which the content packets traveled. But in dynamic routing, cache and mirror
servers are used, so the content delivery route is notconstant. When QoS degradation is detected, it isnecessary to specify the distribution server. Then the
degraded-QoS distribution route from the distribution
server to the active agent is specified by using routing
information and traceroute information.
5.2 Localizing the degraded-QoS section
Next, we determine whether the degraded-QoS section is in the iDC or in the network. If the primarycause is the performance of the content server, thiscan be readily determined simply by scrutinizing theserver log, so the first step is to analyze the server log.
But if the source is in the network, the localization
granularity varies depending on the network’s configuration. For example, a provider can perform muchfiner-grained localization and narrow down to muchsmaller sections on networks under its own control
than on networks controlled by other providers.
Localization of sections is done by active measurement using packet losses, throughput, and otherdegradation criteria (Fig. 4).
5.3 Localizing the degraded-QoS equipment
The next step is to localize the equipment causing
the QoS degradation in cases where the equipment ison the provider’s own network. MIB data is collectedfrom equipment (routers and other equipment having
MIBs) on sections localized in the previous step, andbottlenecks are pinpointed using the packet loss ratio,line activity ratio, and other degradation criteria (Fig.
5).
6. QoS perceived by each user
When providers receive inquiries or complaints

28

NTT Technical Review

Selected Papers

User
Interconnected networks

iDC

Provider’s own network

Content server
(Log)

PC
Traffic generator
Activeagent

Performancemeasurement
section

Performancemeasurement
sectionActiveagent

Activeagent

Activeagent
Management
serverFig. 4. Scheme for localizing degraded sections.

User
Interconnected networks

iDC

Provider’s own network

Server
(Log)

PC

(MIB)

(MIB)

Traffic generator
MIB data
collected

Activeagent

Management
serverFig. 5. Scheme for pinpointing bottlenecks.

Type 2: Information fromthe content serverType 1: Informationfrom the terminalInterconnected networks

Type 3: Information from the
network equipment
Provider’s own network

iDC

Content
serverLayer-7switchFirewallFig. 6. Possible deployments of passive agents.

about the picture and sound quality of content, theymust verify the claims, identify the cause of the problem, and respond to the customer. In some cases, thecause will be the user’s terminal, but in other cases itwill be the network.
To grasp the distribution QoS delivered to eachuser, it is not enough to know the network QoS. If
packet retransmission and other control capabilities
are supported in the application layer, then detailed
QoS information can be obtained by passive agents.
There are three possible ways of deploying passive
agents, as shown in Fig. 6. Here, we briefly consider
Vol. 4 No. 2 Feb. 2006

the features of each type.
• Type 1: A passive agent is installed on the user’s
PC. Its software collects the necessary QoS-related information. This permits the collection of
information in the application layer, but if theinformation and the acquisition method dependon the distribution application, then different
agent software may be required for each distribution application.
• Type 2: Passive agents are installed in the content
server in the iDC. Quality information is collected using the log information stored in the content
29Selected Papers

distribution server (access and application logs).
This permits state management of all users
accessing the server, but has a similar problem toType 1: if the application log is used, the log
information could differ with every distribution
application.
• Type 3: Passive agents are installed in network
equipment such as layer-7 switches. An interface
is specified between the passive agent and network equipment, and higher-layer information iscollected and aggregated by an external device.
This enables the status of applications to be
tracked while avoiding bottlenecks.
These deployment types are compared for one
agent in Table 2. Type 1 enables us to acquire the
QoS data for each user individually. With Types 2 and3, we must extract each user’s QoS data from the QoS
data of all users accommodated by the content serverand by the network equipment, respectively. Therefore, we chose Type 1 considering the volume of data
and the computational complexity of collected QoSdata for our system.

Table 2. Comparison of passive agents.
Volume of data(by one agent)

Data acquisition
cycle (minimum)

Computational
complexity

Type 1Small

A few seconds

Small

Type 2

Large

Every 8 kilobytes

Large

Type 3Large

A few seconds

Large

In this section, we show an example that compares
the use of active measurement with passive measurement for evaluating QoS criteria. In the Type 1 passive measurement described in the previous section,
application-layer QoS data is obtained from the various users’ PCs, and the QoS criterion can be represented by
Quality = (r + rr)/(r + rr + PacketLost) × 100, (2)where r is the number of received packets, rr is thenumber of received retransmitted packets, and PacketLost is the number of packets not restored either by
error correction in the application layer or by retransmission in the network layer.
Figure 7 shows measurement results when an
active agent was deployed near the user and a passive
agent was installed on the user’s PC. Pr (defined by
Eq. (1)) is the packet loss rate in the IP layer. Event 1
represents the situation where packets were continually being lost, and the Quality value was declining
due to distribution application buffering and timeouts. Good agreement was obtained between the values measured by the active and passive agents. On theother hand, Event 2 is the situation where packet loss
occurred in the network layer, but the Quality in theapplication layer fell off precipitously. Even though
packet loss occurred, the Quality value did not
decline as much as in Event 1 because the packets

Win Dump

Pr

Active agent

11:06:43

11:08:10

11:09:36

Event 1

11:12:29

11:13:55

Event 2200
150

Quality

120
100
80
60
40
20
0
11:05:17

11:11:02

Quality

100

PacketLost50

11:06:43

11:08:10

11:09:36

11:11:02

11:12:29

PacketLost

10
8
6
4
2
0
11:05:17

7. Comparison of QoS evaluation by active andpassive measurement0
11:13:55

Fig. 7. Quality criteria (Pr, Quality, and PacketLost ).

30

NTT Technical Review

Selected Papers

were restored by the retransmission capability of the
distribution application.
8. Conclusion
After reviewing the requirements for QoS management for video distribution services, we described our
QoS management system, which detects degradationin user-perceived QoS by active measurement andlocates its cause and evaluates the QoS perceived byeach user by passive measurement. We assumed thatthe system’s management objects were 200 active
agents, 200 distribution servers, and 1000 routers.
We described how to detect degradation by active
measurement and locate its cause. Moreover, we tested a system as an example of QoS evaluation by
active and passive measurement. If the application
supports retransmission and other control capabilities, the combination of active and passive measurement in the application layer provides a more accurate grasp of the distribution QoS and the users’ perceived QoS.
The method of detecting QoS degradation should
consider the influence of active measurement on the
network. In addition, it is important to understand thecharacteristics of the distribution application and to
decide how to arrange the measurement points andset the measurement period. Moreover, it is important
to use information obtained from the network element management system to collect network composition and status information effectively. We will
investigate these points in future.

Akihiko Tsuno
Engineer, Network Software Service Project,
NTT Network Service Systems Laboratories.
He received the B.E. and M.E. degrees in information computer engineering from Niigata University, Niigata in 1999 and 2001, respectively.
He joined NTT in 2001. Since then, he has been
engaged in research on quality management systems. He is currently engaged in the developmentof session control software.

Yoshihiro Otsuka
Senior Research Engineer, Supervisor, Network Software Service Project, NTT Network
Service Systems Laboratories.
He received the B.E. degree in electrical andelectronic engineering from Kanazawa University, Ishikawa in 1983 and the M.E. degree in physical electronics from Tokyo Institute of Technology, Tokyo in 1985. Since joining NTT in 1985,
he has mainly been engaged in research onbroadband switching systems and network management systems. He is currently working on themodeling of network management functions andmulti-layered network management systems. He
is a member of the Institute of Electronics, Information and Communication Engineers (IEICE)of Japan.

Souhei Majima
Senior Research Engineer, Supervisor, Network Software Service Project, NTT Network
Service Systems Laboratories.
He received the B.E. and M.E. degrees in information science from Fukui University, Fukui in
1982 and 1984, respectively. He joined NTT
Laboratories in 1984. He has been involved in
the development of a computer-aided softwaredevelopment environment (CASE) system and aswitching equipment management system forlarge-scale networks. He is currently engaged inresearch on an NGN operation support system.
He is a member of IEICE and the Information
Processing Society of Japan.

References
[1]

[2]

[3]

T. Sakurai, S. Majima, M. Yokoyama, and H. Tokunaga, “Study of
QoS management architecture in the multimedia services era,” Technical Proceedings of APNOMS 2001, Sydney, Australia, Sep. 2001.
M. Brodie, I. Rish, and S. Ma, “Optimizing Probe Selection for Fault
Localization,” 12th IFIP International Workshop on Distributed Systems: Operations & Management DSOM2002, Nancy, France, October 15-17, 2001.
J. Padhye, V. Firoiu, D. Towsley, and J. Kurose, “Modeling TCP
Throughput: A Simple Model and its Empirical Validation,” Proceedings of ACM SIGCOMM98, Sep. 1998.

Vol. 4 No. 2 Feb. 200631