NTT Technical Review
May 2013 Vol. 11 No. 5
Feature Articles: QoE Estimation Technologies
Toward QoE-centric Operation of
Telecommunication Services
Monitoring the Quality of IPTV Services
Playback State Estimation of
Progressive Download-based Video Services
Performance Estimation Techniques forBrowser-based Applications
QoE Assessment Methodologies for
3D Video Services

Regular Articles
Historical Overview of Semiconductor Device Reliability
for Telecommunication Networks––
Field Data, Prediction Model of Device Failure Rate,and Wear-out Failure Analyses at NTT

Global Standardization Activities
Overview of MPLS-TP Standardization

Information
NTT Establishes NTT Innovation Institute, Inc.
as its New R&D Center in North America

Feature Articles: QoE Estimation Technologies

Toward QoE-centric Operation of
Telecommunication Services
Akira Takahashi and Takanori Hayashi
Abstract

The rapid spread in the use of smartphones and cloud-based services has accelerated the speed ofevolution of telecommunications application software, resulting in a great diversity of such software.
While the quality of experience (QoE) of telecommunications services has been directly related to network performance in the past, the current situation is not that simple. This is because the QoE of telecommunications applications is affected not only by network performance, but also by server and terminal
performance to a great extent. In this article we propose a framework for improving user QoE, and we
introduce recent research and development activities underway at NTT intended to achieve this goal.

1. Quality of experience (QoE)
and network performance
All of the factors affecting the end-to-end quality of
conventional public switched telephone network(PSTN) services are under the control of telecommunications carriers except for telephone terminals. The
characteristics of terminals are clearly defined instandards and in domestic laws, for example, theRegulations for Telecommunications Facilities for
Telecommunications Business. Therefore, if the
PSTN network is confirmed to be operating normally,
it is very rare for an end user to encounter a serious
problem. This implies that QoE ≈ (is approximately
equal to) network performance holds in the PSTN
world.
When the Internet era arrived, the variation in terminals drastically increased. At that time, terminals
consisted mainly of personal computers (PCs). However, in the 1990s, the primary factor affecting theQoE of web-browsing was network speed, referred toas throughput, since network capacity was limited.
The relationship QoE ≈ network performance wasstill maintained.
With the arrival of the broadband era in the late
1990s, the behavior of telecommunications application software had become increasingly complex. This
resulted in a change in the dominant factors of QoE;
that is, the server and terminal performance seriously

affects QoE. For example, the video quality of Internet protocol television (IPTV) is dependent on terminal technologies such as forward error correction. In
cloud-based applications, the server mainly transmits
executable programs, and the terminal executes themin order to create and render the content. This meansthat PC performance is more important than network
speed. Moreover, the use of wireless local area networks and power-line communication in home networks has brought new quality degradation factorsthat cannot be monitored or controlled by network
carriers.
The above-mentioned examples reveal that the relationship QoE ≈ network performance no longer
holds. What makes the situation more complicated isthat even network operation has become difficult inan end-to-end sense due to the multi-provider environment (Fig. 1).
If we take all of these emerging factors into account,
we can conclude that the first step towards maximizing user QoE is to quantify and visualize it.
2. QoE-centric operation
As a telecommunications carrier, NTT has madeevery effort to maintain and improve network performance through daily network operations. However,
due to changes in the balance of quality factors, there
are quite a few problems that cannot be found and
NTT Technical Review

Feature Articles

Appliances

Telephone

Storage server
IP transmitter

PersonalcomputerTV

Network
Encoder
Shooting/recording
quality
STB: set-top box

CodingqualityHomenetwork

STB

Transmission Home- Decoding Performance of display
network
Usability of controller
qualityqualityquality
Functionality of menu

Conventional network operation

Fig. 1. Factors affecting QoE of IPTV.

solved through network operation only. How can a
network carrier solve the problems in the following
examples?
2.1 Example 1
Bob likes watching movies. He was interested in
the video on demand menu of an IPTV service. He
decided to subscribe to the IPTV service and to a
Next Generation Network (NGN) service as well. He
placed the set-top box (STB) in his living room,
although the home gateway was placed in the pantry,
where the former PSTN line was installed. It was difficult to lay Ethernet cable from there to the livingroom, so he bought a Wi-Fi router, on whose box was
the claim “300 Mbit/s Wi-Fi! No degradation even for
HD (high definition) videos!”In the beginning, he enjoyed watching movies via
the NGN. However, he began experiencing serious
video degradation that continued to increase, and thevideo sometimes froze. He did not notice—which
was not surprising—that the Wi-Fi router and thedigital cordless telephone that he had recently bought
used the same frequency range and were interferingwith each other.
He was too angry about the quality to think of calling the customer support center. Rather, he remembered the flyer about the cable TV that the neighborhood self-governing body had recommended, and he
immediately called the cable company. Then, he cancelled the IPTV service and at the same time cancelled the NGN service because the cable company
salesman had suggested that it would be much cheapVol. 11 No. 5 May 2013

er to use the cable network for his telephone andInternet service.
The cable company laid the cable directly to the
STB in the living room. Thus, Bob did not use the
Wi-Fi router. Naturally, he experienced no degradation and therefore believed that cable TV was muchbetter than IPTV. Since then, Bob talks about his
experience to his friends whenever he has a chanceand stresses the superiority of cable TV over IPTV.
2.2 Example 2
A corporate customer (Company A), was planning
to replace the business applications they were using.
At the same time, they were considering the possibility of using the cloud environment provided by atelecom operator (Operator B) in order to reduce
costs. However, they were worried about quality degradation such as response latency in using the application software. Therefore, they asked a sales engineer of Operator B about this and were told that
Operator B had carried out a complete performance
evaluation and found no problems.
After a one-month trial by the telecommunications
service division of Company A, they decided to adopt
the proposal by Operator B since they had found no
performance problems. They then extended the trial
to all their divisions.
A couple of weeks later, the telecommunications
service division started receiving many complaints
from several locations about the very slow responseof the application software. The person in charge ofthe telecommunications service division in Company
Feature Articles

A did not experience any degradation, so he asked
Operator B to check the transmission performance ofthe virtual private network (VPN) provided by Operator B. Operator B enhanced the bandwidth of theVPN, and the performance improved in some locations. However, most users still experienced serious
delays in using the application software.
Finally, because of this problem, Company A
decided not to accept Operator B’s proposal when thetrial was finished. Nobody knew that many users in
Company A had installed a certain kind of software in
the background, and this software wasted CPU (central processing unit) power, resulting in poor rendering performance for browser-based applications.
Although the above examples are fictitious, they
could actually occur. The issue here is that it is
becoming increasingly difficult to understand user
dissatisfaction based only on conventional network
management. We often take a questionnaire approachto determine QoE problems. However, this takes a lotof time to solve the problems. Thus, we have been
studying methodologies for estimating user QoE
based on parameters that can be observed by service/
network providers. These technologies will contribute to the operation of telecommunications services
based on user QoE, which we call QoE-centric operation.

The second article, entitled “Playback State Estimation of Progressive Download-based Video Services” [2], introduces a similar technology for TCP(transmission control protocol)-based progressivedownload applications such as BeeTV, which is provided by NTT DOCOMO, and YouTube. The method
developed at NTT intelligently exploits the mechanism of TCP and achieves QoE monitoring at any
point between a server and terminal.
The third article, entitled “Performance Estimation
Techniques for Browser-based Applications” [3],describes techniques that enable QoE monitoring ofbrowser-based applications, which are widely used incloud-based environments. These techniques make itpossible to estimate the waiting time for complex
content that uses recent web technologies such as
Ajax and Flash.
Finally, in the fourth article, “QoE Assessment
Methodologies for 3D Video Services” [4], we introduce techniques to evaluate the quality of 3D video
and also describe their objective estimation methodology. In evaluating 3D video, it is important to quantify the degree of fatigue, for example, in addition to
the conventional picture clarity.

3. QoE estimation technologies

Up to this point, we have emphasized the importance of evaluating/estimating QoE. It is clear that
evaluation/estimation is the first step toward realizing
QoE-centric operation, for which we also need todevelop methodologies for mapping network, server,
and terminal performance onto QoE. In addition, it isimportant to develop appropriate customer relations
management policies.
The concept of QoE-centric operation is a framework for providing and maintaining good telecommunications quality. We also propose a framework inwhich network providers improve user QoE in collaboration with service providers and end users. This
is not the conventional one-way quality given by network providers, but a mutual effort with service providers and end users (Fig. 2). We call this Co-creation
Quality.
Obtaining information on the current quality andpossible alternatives is essential in order to encourage
users to take action to improve their QoE. For example, users of public transportation such as trains
expect the trains to run exactly on schedule. By contrast, people who drive their own cars need information on traffic jams and weather forecasts to help

These Feature Articles introduce QoE estimation
technologies with which we can estimate user QoE
based only on observable parameters such as packetheader information. Conventionally, such technologies were assumed to be applied to off-line quality
measurement, and therefore, the media-signal-based
approach was taken to analyze speech waveforms or
pixel data of videos. This approach requires an algorithm capable of handling a very heavy and complex
computational load. However, to enable online realtime monitoring of QoE, passive measurement and alight-weight computational load are necessary for
estimating QoE.
The first article in this issue, which is entitled“Monitoring the Quality of IPTV Services” [1], introduces a QoE estimation methodology for UDP (userdatagram protocol)-based IPTV. The validity of thistechnology was thoroughly evaluated by ITU-T(International Telecommunication Union, Telecommunication Standardization Sector) and was standardized as ITU-T Recommendation P.1201.1 in2012.


4. Expanding horizons ofservice-quality research

NTT Technical Review

Feature Articles

Report on service quality
Are users able tocomfortably use
applications?

Quality announcement and forecast

Ten percent of your customers
are experiencing bad quality
services due to terminal
problems.

Quality
information

xSP

We want to control
applications optimallyaccording to the
network conditions…

Networks are crowded inthis area. Throughput isapproximately 200 kbit/s.

Quality
information

NTT

We can regularly input
information about the indicatorssuch as RTT or a throughput toyour applications.

My cell phone reception
shows three bars, so why is
the network so slow?

LTE networks are crowded now,
so you can use voice but notvideo calls. On Wi-Fi networks,
you can use video calls.

Users

I want to make atelephone call. Can I
use video?

Network/service recommendation

Feedback to application control
LTE: Long Term Evolution
RTT: round trip timexSP: any type of service provider

Fig. 2. Concept of Co-creation Quality.

RequestNetwork
providerf(user_ID=1, 2, 9 …)

xSP

Quality information# of users

user_ID = 1, location_info.,
application_info., transmission_quality,
application_quality, estimated_QoE…
user_ID = 2, location_info.,
application_info., transmission_quality,
application_quality, estimated_QoE…

Reporting the current status
and analyzing the cause of
the problem

Threshold

Good

Bad

Video streaming quality

Fig. 3. Quality API.

Vol. 11 No. 5 May 2013



Feature Articles

them decide their route and schedule. Similarly, we
believe network providers should provide more information on quality to users and/or service providers.
To do this, technologies for monitoring, predicting,and visualizing information on quality are necessary.
Such information could be used to improve customer support by service providers. For example, if aservice provider can obtain the current QoE information of their end users from a network provider simply by requesting it with a user ID, they can recommend an alternative to the end user when quality was
low before receiving a complaint from the user
(Fig. 3). It may also be possible to optimize thebehavior of application software so that it adjusts tocurrent and future quality conditions of networks. We
call this kind of network function a Quality API(application programming interface) and consider itpart of the network API.
The technologies introduced in this article are fundamental to achieving QoE-centric operation, Co-

Akira Takahashi

Manager of the IP Service Network Engineering Group, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the B.S. degree in mathematics
from Hokkaido University in 1988, the M.S.
degree in electrical engineering from California
Institute of Technology, USA, in 1993, and the
Ph.D. degree in engineering from the Universityof Tsukuba, Ibaraki, in 2007. He joined NTT in1988 and has been engaged in the quality assessment of audio and visual communications. He
was a co-Rapporteur of ITU-T Question 13/12 on
Multimedia QoE and its assessment during the
2004–2008 Study Period. He is a Vice-Chairmanof ITU-T Study Group 12 (SG12) for the 2009–2012 and 2013–2016 Study Periods. He is aVice-Chairman of the Technical Committee ofCommunication Quality in the Institute of Electronics, Information and Communication Engineers (IEICE). He received the Telecommunication Technology Committee Award in Japan in
2004 and the ITU-AJ Award in Japan in 2005. Healso received the Best Tutorial Paper Award fromIEICE in Japan in 2006 and the Telecommunications Advancement Foundation Award in Japan
in 2007 and 2008.



creation Quality, and Quality APIs in the future. NTT
R&D (research and development) will continue suchstudies in the future.
References
[1]

[2]

[3]

[4]

K. Yamagishi, N. Egi, and T. Tominaga, “Monitoring the Quality of
IPTV Services,” NTT Technical Review, Vol. 11, No. 5, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
05fa2.html
H. Honda, H. Yamamoto, S. Nakamura, and A. Takahashi, “Playback
State Estimation of Progressive Download-based Video Services,”
NTT Technical Review, Vol. 11, No. 5, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
05fa3.html
H. Yamamoto, S. Nakamura, H. Honda, and A. Takahashi, “Performance Estimation Techniques for Browser-based Applications,” NTT
Technical Review, Vol. 11, No. 5, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
05fa4.html
K. Yamagishi, T. Kawano, and K. Kawashima, “QoE Assessment
Methodologies for 3D Video Services,” NTT Technical Review, Vol.
11, No. 5, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
05fa5.html

Takanori Hayashi

Manager of the Service Assessment Group,
Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees
in engineering from the University of Tsukuba,
Ibaraki, in 1988, 1990, and 2007, respectively.
He joined NTT in 1990 and has been engaged in
the quality assessment of multimedia telecommunication and network performance measurement methods. He received the Telecommunication Advancement Foundation Award in Japan in
2008.

NTT Technical Review

Feature Articles: QoE Estimation Technologies

Monitoring the Quality of
IPTV Services
Kazuhisa Yamagishi, Noritsugu Egi, andToshiko Tominaga
Abstract

This article introduces a quality-monitoring model that can be used to estimate end-users’ quality ofexperience of Internet protocol television services using packet header information. This model can beused for in-service quality monitoring.

1. Introduction

processing chain consists of video acquisition andediting, encoding, network transmission, decoding,and display, as mentioned in section 1. All of theseelements can affect end-users’ QoE. ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) Recommendation
G.1081 [1] defines performance monitoring points
for IPTV services that will enable the service provider and/or network operator to monitor the performance of the entire IPTV service delivery process.
There are five points in the processing chain where
performance quality is monitored: the source media
and metadata are monitored at point 1; encoded andpacketized source media are monitored at point 2;
packet transmission characteristics are monitored atpoint 3; received packets are monitored at point 4 todetermine whether they can adequately provide therequired QoE at the client terminal, and the displayed
media are monitored at the client terminal at point 5.
If all the information obtained at these monitoring

Internet protocol television (IPTV) services are
now widely provided. The quality of experience(QoE) of IPTV services is affected by the audiovisualcontent, encoding and decoding techniques, network
performance, and display technology. It is therefore
important to ensure that end-users receive high-quality IPTV content. The ideal way to do that would be
for the service provider to identify the quality degradation factors, monitor QoE by taking the main quality degradation factors into account, and finally, toconduct a thorough investigation to determine thecause of quality degradation and then address the
problems.
2. Activities of ITU-T SG12
in quality monitoring
An IPTV processing chain is shown in Fig. 1. ThisHome
gateway

Acquisition

Editing

Encoder

Point 1

Point 2

Core network

Access network

Point 3

Point 4

Set-topbox

Decoder/display

Viewing

Point 5

Fig. 1. IPTV processing chain.

NTT Technical Review

Feature Articles

points is integrated, the locations where quality has
degraded can be determined.
ITU-T Recommendation G.1081 does not define
how objective quality assessment models should beapplied at each performance monitoring point. Monitoring the QoE at the head end is important because
quality degradation at points 1 and 2 influences the
QoE of all users. In these cases, full reference (FR)media-layer models are suitable for monitoring quality. At points 3 and 4, it is preferable to analyze packets using a method with low computational load
because the analysis needs to be implemented in terminals such as mobile terminals, home gateways, andset-top boxes (STBs). In these cases, packet-layer
models are suitable for estimating QoE from IP
packet header information. At point 5, it is essential
to monitor the QoE by using the no-reference (NR)media-layer model, which estimates QoE from media
signals received at the client terminal. Because FR
media-layer models [2]–[4] that can be applied tohead-end QoE monitoring have already been standardized, NTT has focused on developing packetlayer models that can be used for end-user QoE
monitoring using packet header information [5].
Application areas can be categorized into lower resolution (LR: quarter common intermediate format(QCIF), quarter video graphics array (QVGA), andParameter extraction
module for video

Packet header
information
RTP header

half VGA (HVGA)) and higher resolution (HR:standard definition (SD) and high definition (HD))
areas. The LR application area can be used for mobile
IPTV services, and the HR application area can beused for STB based IPTV services. In 2006, ITU-T
study group 12 (SG12) also launched a project thataims to develop quality monitoring models for LR
and HR application areas. In October 2012, ITU-T
standardized two Recommendations; ITU-T Recommendation P.1201.1 [6] can be used for the LR application area, and P.1201.2 [7] can be used for the HR
application area. This article introduces ITU-T Recommendation P.1201.1 as it is most relevant becauseof the recent rapid growth of Mobile IPTV services.
The P.1201.1 model was developed by integrating the
NTT and Huawei models and was verified in an
ITU-T performance evaluation contest.
3. P.1201.1 model
A block diagram of the P.1201.1 model is shown in
Fig. 2. This model consists of three modules:
parameter-extraction, parameter-calculation, andquality-estimation modules as follows.
1) Parameter-extraction moduleThis module extracts a Real-time Transport Protocol (RTP) timestamp, sequence number, and marker

Parameter calculation
module for video

Video quality
estimation module

Video MOS

Audiovisual quality
estimation module

Parameter extraction
module for rebuffering

UDP header
IP headerSide
information
about codec,
client behavior,etc.

Parameter extraction
module for audio

Parameter calculation
module for audio

Available to all modules

Audio quality
estimation module

Audio MOS

MOS: mean opinion score
UDP: user datagram protocol

Fig. 2. Block diagram of P.1201.1 model [5].

Vol. 11 No. 5 May 2013



Feature Articles

bit from RTP headers and the rebuffering startingtime and length from the client terminal.
2) Parameter-calculation moduleThis module calculates audio- and video-related
quality parameters (e.g., coding bit rate and packet
loss) using parameters extracted by the parameterextraction module.
3) Quality-estimation moduleThis module estimates audio, video, and audiovisual quality using quality parameters calculated by
the parameter-calculation module. Additionally, each
module has a sub-module, as shown in Fig. 2.
This article summarizes the quality degradation
factors processed by the quality-estimation modules.
The details of the parameter extraction and calculation modules are not explained here, as they arebeyond the scope of this article.
3.1 Audio quality estimation module
Audio quality is affected by the codec (coderdecoder) type, coding bit rate, packet loss, and rebuffering, so it is necessary to model the relationshipbetween the following quality factors and the subjective audio quality.
-	Effect of audio codec (i.e., AMR-NB (adaptive
multi-rate narrowband), AMR-WB+ (extended
adaptive multi-rate wideband), AAC-LC(advanced audio coding low complexity), HEAACv1 (high-efficiency AAC, version 1), andHE-AACv2) on audio quality- Effect of coding bit rate on audio quality-	Effect of lost audio frame length due to packet
loss on audio quality-	Effect of rebuffering on audio quality (understudy)
3.2 Video quality estimation module
As with audio quality, video quality is affected by
the codec type, coding bit rate, packet loss, andrebuffering. Video quality is also affected by the
number of bits per video frame type because it varies

I-frameP-framedepending on the spatio-temporal information of thevideo content. Therefore, it is necessary to model the
relationship between the following quality factors
and the subjective video quality.
-	Effect of video codec (i.e., MPEG-4 (Motion
Picture Experts Group, version 4) and H.264) on
video quality-	Effect of video resolution (i.e., QCIF, QVGA,
and HVGA) on video quality-	Effect of coding bit rate, frame rate, and ratio of
I-frame (intra-coded frame) bit count to the total
bit count on video quality-	Effect of the number of packet-loss events, number of damaged video frames, and the video
frame area damaged by the packet losses onvideo quality-	Effect of the number of rebuffering events, average rebuffering length, and average interval
between rebuffering events on video quality
3.3 Audiovisual quality estimation module
Audiovisual quality is estimated based on audioand video-related quality factors since audiovisualquality is affected by both audio and video quality.
Packet headers do not indicate the codec type andimplementation or the video resolution, so coefficients for audio, video, and audiovisual quality estimation modules need to be optimized for these factors. ITU-T Recommendation P.1201.1 provides the
optimized coefficients [6].
The rest of this section describes how the number ofdamaged video frames and the damaged video frame
area are derived. When a video frame is lost, degradations of the video frame are propagated until the next
I frame is received (Fig. 3). Therefore, determining
the number of damaged video frames is an effective
way of estimating quality. In addition, the damaged
video frame area itself affects video quality. The
damaged area is derived as follows: 1) the second
packet of the “i+1”th video frame is lost, so the mod-P-frame“i+1”th video frame “i+2”th video frame“i”th video frame

Received packet

Lost packet

P-frameI-frame“i+3”th video frame“i+4”th video frame

Lost packet due to propagation

Fig. 3. Number of damaged video frames and damaged video frame area [5].

NTT Technical Review

Feature Articles

ule outputs 50% as the damaged video frame area;
2) the first packet of the “i+2”th video frame is lost,
so the module outputs 100% as the damaged video
frame area; 3) the “i+3”th video frame has not lostany packets, so the module outputs 100% as the damaged video frame area because the previous video
frame was lost, and degradation propagation lasts
until the next I-frame.
4. Performance of P.1201.1 model
ITU-T SG12 verified the validity of the P.1201.1
model by conducting numerous subjective tests using
audio, video, and audiovisual sequences that weregenerated by varying the codec type, coding bit rate,
packet-loss pattern, packet-loss concealment, andrebuffering pattern. The audio, video, and audiovisualquality estimation modules were verified by using theroot mean square error (RMSE) and Pearson’s correlation (PC). Table 1 (see Appendix of ITU-T Recommendation P.1201 [8]) lists RMSE and PC values.
Additionally, ITU-T verified that the P.1201.1 model
reached a sufficient level of quality-estimation accuracy because the RMSE was small and PC was high,as listed in Table 1.

Table 1. Performance of P.1201.1 model [7].
RMSE

PC

Audiovisual

0.470

0.852

Video

0.535

0.830

Audio

0.351

0.941

6. Conclusion
This article introduced a quality estimation model
we developed for IPTV services and the P.1201.1
model. The P.1201.1 model has reached a sufficientlevel of quality-estimation accuracy and can beapplied to monitor the quality of mobile IPTV. As
described previously, the P.1201.1 model cannot be
used for transmission control protocol (TCP)-based
progressive video streaming because it was developed for realtime IPTV services. Although there aredifferent types of video streaming, quality factors
such as coding and rebuffering in realtime video
streaming are the same as those of TCP-based video
streaming. Therefore, it may be possible to extend themodel and apply it to TCP-based video streaming if
the parameter-extraction module can process TCP
headers. Although the application area of the P.1201.1
model is limited to lower resolution, it is desirable toextend the model to the higher resolution application
area because the newer mobile terminals support
higher resolution. ITU-T SG12 is planning to extend
the P.1201.1 model because of these technologicaldevelopments.
In addition to promoting the P.1201.1 model, towhich NTT contributed, our group is involved in
globally competitive research and development of aquality-monitoring technique for a new service.
References
[1]
[2]
[3]

5. Application scenario[4]In-service quality monitoring is an important application of the P.1201.1 model. The P.1201.1 model canbe applied as followed:
1)	Customer complaints are often made becauseof quality degradation. Service providers canresolve them by monitoring the quality level.
Even if the end-user does not notice any degradation, service providers can resolve problems
due to quality degradation before customers
complain about the quality.
2)	Quality degradation locations can be detected
quickly by gathering data on quality and thecauses of quality degradation from manyusers.
Vol. 11 No. 5 May 2013

[5]

[6]
[7]
[8]

ITU-T Recommendation G.1081, “Performance monitoring points
for IPTV,” Oct. 2008.
ITU-T Recommendation J.341, “Objective Perceptual Multimedia
Video Quality Measurement of HDTV for Digital Cable Television in
the Presence of a Full Reference,” Jan. 2011.
ITU-T Recommendation J.144, “Objective Perceptual Video Quality
Measurement Techniques for Digital Cable Television in the Presenceof a Full Reference,” Mar. 2004.
ITU-T Recommendation J.247, “Objective Perceptual Multimedia
Video Quality Measurement in the Presence of a Full Reference,”
Aug. 2008.
K. Yamagishi, K. Ushiki, T. Hayashi, and A. Takahashi, “Packet-layer
Model for End-user QoE Management,” NTT Technical Review,Vol. 7, No. 4, 2009.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr200904sf6.html
ITU-T Recommendation P.1201.1, “Parametric non-intrusive assessment of audiovisual media streaming quality––lower resolution application area,” Oct. 2012.
ITU-T Recommendation P.1201.2, “Parametric non-intrusive
assessment of audiovisual media streaming quality––higher resolution application area,” Oct. 2012.
ITU-T Recommendation P.1201, “Parametric non-intrusive assessment of audiovisual media streaming quality,” Oct. 2012.

Feature Articles

Kazuhisa Yamagishi

Research Engineer, Service Assessment Group,
Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
He received the B.E. degree in electrical engineering from Tokyo University of Science andthe M.E. degree in electronics, information, andcommunication engineering from Waseda University, Tokyo, in 2001 and 2003, respectively.
He joined NTT in 2003. He has been engaged insubjective quality assessment of multimedia
telecommunications and image coding. He iscurrently working on quality assessment of multimedia services over IP networks. He has been
contributing to ITU-T SG12 since 2006. From2010 to 2011, he was a visiting researcher at
Arizona State University. He received the Young
Investigator’s Award from the Institute of Electronics, Information and Communication Engineers (IEICE) in 2007 and the Telecommunication Advancement Foundation Award in Japan in
2008. He is a member of IEICE.

Toshiko Tominaga

Research Engineer, Service Assessment Group,
Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
She received the B.E. and M.E. degrees inelectrical engineering from the University ofElectro-Communications, Tokyo, in 1987 and1989, respectively. She joined NTT in 1989 andhas been engaged in research on facsimile andvideo quality assessment. She is currently working on the quality assessment of IPTV and web
services. She received the Telecommunication
Advancement Foundation Award in Japan in
2008. She is a member of IEICE.

Noritsugu Egi

Researcher, Service Assessment Group, Communication Traffic & Service Quality Project,
NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in electrical communication engineering from Tohoku
University, Miyagi, in 2003 and 2005, respectively. Since joining NTT in 2005, he has been
engaged in research on speech and audio quality
assessment. He is a member of IEICE.

NTT Technical Review

Feature Articles: QoE Estimation Technologies

Playback State Estimation of
Progressive Download-based
Video Services
Hirotada Honda, Hiroshi Yamamoto, Sorami Nakamura,and Akira Takahashi
Abstract

The recent proliferation of smartphones and tablet terminals has resulted in the rapid development ofvideo applications. However, in wireless access lines such as those for LTE (Long Term Evolution) and3G (third-generation) services, packet loss rates and variations in the delay are larger than those in fixedaccess lines. As a result, video services implemented in TCP (transmission control protocol), known as
progressive download (PDL)-based video services, are especially pervasive on wireless access lines.
In this article, we introduce a method of playback state estimation of PDL-based video services. Thismethod makes it possible to accurately estimate the playback state, which will improve the efficiency ofmonitoring the quality of video services in network and service operation processes.

1. Introduction
The use of smartphones and tablet terminals has
increased substantially recently, and with thisincrease, more and more applications are now being
provided via the Internet. In particular, video applications are remarkably pervasive; these are technicallydivided into three categories according to their delivery mechanism: progressive download (PDL)-based
video, download type video, and real-time streaming.
The first two are implemented in TCP (transmission
control protocol) and the third in UDP (user datagram
protocol). PDL-based video services account for
about 40% of the volume of domestic Internet traffic.
Representative PDL-based video services such as
YouTube and Hulu are targeted to users of personal
computers, smartphones, and tablet terminals thataccess the services over the Internet. In addition,some broadcasters also provide PDL-based video
services that enable users to access their news andarchived content. NTT Plala started Hikari TV (television) dokodemo/mobile in 2011, NTT WEST Corporation provides the network platform for SKY PerfecTV! on Demand, and NTT DOCOMO offers


BeeTV and the d-market video store. The NTT Group
believes that as a network and service provider, one ofits most important tasks is improving the quality ofthese services.
The reasons for this are as follows. First, for network providers, PDL-based video services are considered to be the benchmark of the network. Lowquality PDL-based video services will directly lead toa decrease in customer satisfaction. Second, for aservice provider, attractive content and reasonable
prices are of course important, but so is the quality ofthe service itself. Nevertheless, there are no methods
for either continuously monitoring the service quality
or comparing the service quality with other competitive services. Thus, we have developed a method for
addressing these issues for the network and service
providers in the NTT Group.
2. Objectives of proposed method
The following two issues are considered to be themain objectives of the proposed method:(1) Continuously monitoring the service quality;(2) Comparing the service quality with other
NTT Technical Review

Feature Articles(1) Stationary monitoring of service quality(application for network providers)(2) Comparison of service quality(application for service providers)YouTubeNTT

NTT network

Other
companies’
networks

The
TheInternet
Internet
ISP-1
Mobile NW_1

Stationary monitoring of service
quality experienced by NTT users

Mobile NW_2

: Packet capture points
Comparison of service quality

Qualitydegradation

Quality transition of NTT network users

Service quality

Frequency ofquality degradation:Packet capture points

ISP-2

Quality of NTT service
Reversalof service
quality

Quality of competitive
services

Churn rate

Packet capturein NTT network

Comparison- For NTT services:
Capture packets
directly beneaththe head end.
- For competitive
services: Capture
packets in the test
terminals of clients(monitored users’
samples)

TheTheInternet
Internet

Competitive services

Churn rate
09:00

12:00

15:00

18:00

21:00

Time

Jan.

Apr.

June

Oct. Month

Comparison of service quality of service providers

Fig. 1. Possible applications of proposed method.

competitive services.
The first item refers to efficiently enhancing service
quality monitoring in the network operation processes, which will enable earlier detection and resolutionof quality degradation incidents. The latter item
implies that information on the actual service quality
is provided, and this information is useful for making
decisions on service strategies (Fig. 1).
3. Principal factor affecting the qualityof PDL-based video services
In this section, we discuss the main factor affecting
the quality of PDL-based video services (Fig. 2). In
real-time streaming applications implemented in
UDP, packet losses directly result in block noise ofvideo images, or they halt the playback. When forward error correction (FEC)*1 is applied, a packet
loss that exceeds the redundancy of the FEC configu*1	 FEC: When data are lost due to bit errors or packet losses, theFEC mechanism enables detection and correction at the clientside by using the redundant data included in packets.

Vol. 11 No. 5 May 2013

ration leads to the block noise or to the halt of playback.
In PDL-based video, on the other hand, the lost
packet will be retransmitted by the server thanks to
the functionality of TCP. If the variation in the packet
arrival interval caused by packet losses or jitter
exceeds a certain level, the video playback will halt
due to buffer starvation. Consequently, playback halt
is the principal reason for quality degradation of
PDL-based video services.
4. Proposed estimation method
4.1 Technical details
The explanation in the previous section indicatesthat it is most important to accurately estimate theplayback state for PDL-based video applications. The
proposed method makes it possible to estimate theplayback state, especially the total halt duration andthe number of playback halt events, by using packet
capture data. While existing methods such as YouTube API (application programming interface) [1]
are applicable only to specific services, the proposed
method is basically applicable to arbitrary PDL-basedFeature Articles

Examples of quality degradation events
Real-time streaming- Video quality degradation- Block noise- Frame skip

Hikari TV/BS retransmission, etc.

Progressive download- - Longer initial buffering time
based video- Playback halt due to rebuffering

Real-time streaming

Representative services

Hikari TV dokodemo/mobile, Hulu, BeeTV

Progressive download-based video

ServerServerPacket loss

Packet loss

UDP

Retransmission

TCP
ClientClientBlock noise

Playback halt

BS: broadcast satellite

Playback

Fig. 2. Factors affecting service quality.

Packet capture
data (input)

Estimated state
Playback state(output)
Amount of datain the buffer

Decoding rate(a) Play-out buffer model

Actual state

HaltThreshold for playback startThreshold for playback halt

0

10

20
30
Time (s)4050(c) Estimation result

Amount of datain the buffer

Playback

Thresholdfor playback startThreshold for playback restartThreshold for playback halt
Time(b) Temporal sequence of data in the play-out buffer

Fig. 3. Input and output of proposed method.

video services. Furthermore, it does not require any
additional implementations to either applications or
terminals, and is achieved with only passive monitoring. These are the main advantages of the proposed

method.
We discuss here the technical details of the proposed
method [2]–[4], which is depicted in Fig. 3. Before
the estimation process is carried out, we first estimate
NTT Technical Review

Feature Articles

parameters of the play-out buffer (Fig. 3(a)) such as
the decoding rate, the thresholds of the playback start,
and the halt and restart. This process is executed inadvance of the estimation process. These parameters
vary because of the different operating systems in client terminals, applications, and video resolution levels, so we have to estimate them through carefulobservation of the playback state and the corresponding packet capture data.
In the estimation process, we first extract the time
sequence of the downloaded data volume in each session by using the TCP header in the packet capture
data. Next, we calculate the amount of temporal datain the play-out buffer (Fig. 3(b)). The playback stateat each measurement time, which is the final outputof this method, is determined by the amount of datain the buffer at each measurement time and thethresholds estimated beforehand (Fig. 3(c)). In somecases, additional analysis of the payload information
can enhance the accuracy of the playback state estimation.

4.2 Verification
We verified the effectiveness of the proposed method by applying it to the PDL-based video services
listed in Table 1. An example of the playback state
estimation is shown in Fig. 4. We evaluated the accuracy of the estimation by calculating the matchingrate*2, which was about 92% in this case.
5. Future work
We plan to further enhance the accuracy of estimation by considering the variable decoding rate if theadaptive video quality transitions take place. We will
also investigate the detection of the pause state initiated by users and the estimation of the pause state
duration [4]. We will continue our research and development of methods to estimate the quality of various
application services.
References
[1]
[2]
[3]

Table 1. List of verified services.

[4]

Verified services
YouTubeYouTube API. http://www.youtube.com/dev.
D. Ikegami, H. Honda, H. Yamamoto, H. Nojiri, and A. Takahashi,“Playing State Estimation Method for Progressive Download Services,” IEICE-CQ2011-59, pp. 91–96, 2011 (in Japanese).
H. Honda, D. Ikegami, and H. Yamamoto, “Estimating Video Playback Quality by Observing TCP ACK Packets,” Proc. of the IEICE
General Conf, B-11-25, Okayama, Japan, 2012 (in Japanese).
H. Honda, D. Ikegami, H. Yamamoto, H. Nojiri, and A. Takahashi,“Playback and Pause State Estimation of Progressive Downloadbased Video Services,” IEICE-CQ2012-30, pp. 77–81, 2012 (in Japanese).

Hikari TVdokodemo/mobile
SKY PerfecTV! on demand*2 Matching rate: In this article, we define the matching rate as thetotal time when the estimated and actual playback states are consistently divided by the total time of the video playback.

d-market (BeeTV)

Playback state

Halt

Hulu

Estimated halt state

Playback

Actual halt state

0

5

10

15

Time (s)Fig. 4. Exampl of estimation result.

Vol. 11 No. 5 May 2013



Feature Articles

Hirotada Honda

Research Engineer, IP Service Network Engineering Group, NTT Network Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees
in science from Keio University, Kanagawa, in
2000, 2002, and 2011, respectively. He joined
NTT in 2002. He is currently investigating the
playback quality estimation of progressive download-based video services. He is a member of theInstitute of Electronics, Information and Communication Engineers (IEICE).

Hiroshi Yamamoto

Senior Research Engineer, IP Service Network
Engineering Group, NTT Network Technology
Laboratories.
He received the B.S. and M.S. degrees in information and computer science from Waseda University, Tokyo, in 1999 and 2001, respectively.
He joined NTT Service Integration Laboratories(now NTT Network Technology Laboratories) in2001. He has been working on the architecture
and performance evaluation of IP networks andweb applications. He is a member of IEICE.



Sorami Nakamura

Research Engineer, IP Service Network Engineering Group, NTT Network Technology Laboratories.
She received the B.S. degree in architecture
and building engineering and the M.S. degree inmathematical and computing sciences from
Tokyo Institute of Technology in 2008 and 2010,
respectively. Since joining NTT in 2011, she has
been working on quality design and managementin networks. She is a member of IEICE and the
Operation Research Society of Japan.

Akira Takahashi

Manager of the IP Service Network Engineering Group, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the B.S. degree in mathematics
from Hokkaido University in 1988, the M.S.
degree in electrical engineering from California
Institute of Technology, USA, in 1993, and the
Ph.D. degree in engineering from the Universityof Tsukuba, Ibaraki, in 2007. He joined NTT in1988 and has been engaged in the quality assessment of audio and visual communications. He
was a co-Rapporteur of ITU-T Question 13/12 on
Multimedia QoE and its assessment during the
2004–2008 Study Period. He is a Vice-Chairmanof ITU-T Study Group 12 (SG12) for the 2009–2012 and 2013–2016 Study Periods. He is aVice-Chairman of the Technical Committee ofCommunication Quality in IEICE. He received
the Telecommunication Technology CommitteeAward in Japan in 2004 and the ITU-AJ Award inJapan in 2005. He also received the Best Tutorial
Paper Award from IEICE in Japan in 2006 andthe Telecommunications Advancement Foundation Award in Japan in 2007 and 2008.

NTT Technical Review

Feature Articles: QoE Estimation Technologies

Performance Estimation Techniquesfor Browser-based Applications
Hiroshi Yamamoto, Sorami Nakamura, Hirotada Honda,
and Akira Takahashi
Abstract

Corporate application services using cloud computing are coming into wide use. These services
include SaaS (Software as a Service), a delivery model for cloud-hosted software applications, and areprovided via networks. Therefore, the state of the network and of the user terminal determines whether
the performance assumed in the application development is achieved. This article describes methods wedeveloped for estimating the waiting time experienced by the user and for determining whether or not adecrease in performance was caused by the user terminal. These methods make it possible to visualize
application performance and to provide support when performance declines in browser-based applications, which are the main type of corporate application services.

1. Introduction
Traditionally, applications installed on terminals
have been the mainstream, and applications with a
web browser user interface started coming into wide
use around the year 2000. Then web browsers gained
the capability to serve as an application execution
platform, which then led to the expanding use of technologies that enhance application interactivity, suchas Ajax* or Flash in around 2007 (Fig. 1).
In conventional applications, web-browser-downloaded content such as HTML (hypertext markup
language) and image files, is rendered and displayedin response to a user operation (hereafter referred toas static content). By contrast, in applications using
technologies such as Ajax, the web browser first
downloads executable code such as JavaScript andthen executes the code in response to a user operation(hereafter referred to as dynamic content). With
dynamic content, most of the processing is performedin the terminal, so application performance is lesssusceptible to network or server performance. However, it is strongly influenced by the terminal processing performance.
*Ajax (Asynchronous JavaScript+XML) enables dynamic updating of part of an otherwise static web page.

2. Performance indicators and problem
with existing monitoring technology
2.1 Application performance metrics
Various indicators can be used to measure browserbased application performance. We used an indicatorthat correlates to the user experience and is based on
the timing from the start of a user operation until thetime the results are displayed. We call this the experienced wait time. The layer model of browser-based
application performance indicators is shown in Fig. 2.
The experienced wait time corresponds to a key quality indicator (KQI), and KQI consists of various key
performance indicators (KPIs) such as the data transfer time and the terminal processing time.
We describe here a specific example of the relationship between these indicators. A schematic representation of signals to be exchanged between the terminal and the server in browser-based applications isshown in Fig. 3. The left and right illustrations arerespective examples for static and dynamic content.
With static content, a hypertext transfer protocol(HTTP) signal is issued synchronously with user
operation and data reception, so HTTP response time,a KPI, and experienced wait time, a KQI, are generally consistent. In the case of dynamic content, theHTTP signal is issued asynchronously with user
NTT Technical Review

Feature Articles

Transition in service style
Installed applications

Browser-based applications
Static content

Dynamic content
Scriptingcode

Scripting
Data size:
Big
Frequency:
High

HTML
Greater
impactStand-alone
processing

Ajax
Flash
HTML5
ScriptingRendering
Terminal
processing:Light
Display
complete

Data size:
Small
Frequency:
Low

Rendering
Terminal
processing:
Heavy

Greater
impactDisplay
complete

Display
complete

Fig. 1. Transition in style of application services.

QoE
Quality of experience
KQI
Key quality
indicator

KPI
Key performanceindicator

Dominant performance factor
in browser-based applications

Experienced
wait time

Data transfer time
HTTP response
time…

Experienced
performance

Waits for
input

Terminal
processing
time
CPU
load

Layout
break

…

API
response
time
Transmission delay

Image
quality

Acoustic
quality…

Delayvariation

Packetloss ratio

Free
memoryAPI: application programming interface
CPU: central processing unit
HTTP: hypertext transfer protocol

Fig. 2. Layer model of application performance.

operation and the processing performed only in theuser terminal such as scripting, so HTTP responsetime and experienced wait time do not correlate inmost cases [1].
The percentage of processing time, in which theterminal is occupied, out of the total experienced wait
time for some sample application operations is shownin Fig. 4. These results show that although there aredifferences between the operations, the terminal processing time is a key factor in the experienced wait
time, and the experienced wait time varies greatly inVol. 11 No. 5 May 2013

different types of terminals even when the operationis the same (e.g., Fig. 4, docoiku old terminal andother terminal). These evaluations show that the terminal has become a key performance factor. Furthermore, existing indicators such as data transfer time
(HTTP response time) and server response time (APIresponse time) are not taken into consideration in theterminal processing factor, so it is not possible tograsp the experienced wait time from those indicators.

Feature Articles(a) Static content
ServerTerminal

APL

Terminal

Content request

Content request

Content response

Content response

Experienced wait time

Experienced wait time(b) Dynamic content

Content request
Content response
Content request

APL

Script request
Script response

ScriptingSession registration
Session token response

Content response
HTTP
response time

ServerHTTP responsetime

API responsetime

API response
time

APL: application programming language

Fig. 3. Correlation of performance indicators.

Application name(Operation name) Server and network factors Terminal factors
Same
operation

Experienced wait time (s)Docoiku (top)*
on old terminal

6.3

Docoiku (top)3.5

Zoho (mail)

8.1

Zoho
(message list)
Zoho (top)

4.2
7.2

iTunes (top)

7.2

Rakuten(Product search)

5.7

Rakuten (top)

10.6
0%

20%

40%

60%

80%

Document download
Stylesheet download
Image download

Server andnetwork factors

ScriptingRendering

Terminal factors

100%*(top): Here, “top” means “top user page of docoiku application”. This graph shows the experienced wait time forseveral user operations. For example “Docoiku (top) on old terminal” indicates a user operation to open the“Docoiku top page”.

Fig. 4. Percentages of operation processing time out of total experienced wait time for different applications.

2.2 Performance monitoring problem
Most existing performance monitoring products
measure the HTTP response time and the API
response time mentioned above. These performanceindicators are suitable for evaluating network performance and server performance, but not for evaluating
experienced wait time. This is because the aforementioned terminal processing time is the key factor in
the experienced wait time, and it is not taken into
account in those performance indicators.
To address this problem, we developed a method toestimate the experienced wait time. This method isintended to close the gap that traditional performanceindicators have in measuring the experienced wait
time. We also developed a method to isolate the primary cause of deterioration. Our methods were developed for browser-based applications with dynamic
content in order to (1) estimate the wait time experienced by the user and (2) determine whether or not adecrease in performance was caused by the terminal.
NTT Technical Review

Feature Articles(2) Detecting terminal bottlenecks(1) Estimating the experienced wait time

Web browser log<Input>Web browser log(in deterioration case)<Input>Our estimation
algorithm

Web browser
log(in normal case)

Our deterioration
determination
algorithm<Output><Output>
Estimation

Estimated wait time

User experience

Experienced wait
time
User operation

Network

APL

What caused
the deterioration?

Display complete
Time

Fig. 5. Overview of estimation methods.

Table 1. Evaluated applications.
Category

Application nameSourceAccuracy (*)

OfficeMicrosoft Office Web Apps
Google Docs
Zoho
Lotus Live

Microsoft
Google
Zoho
IBM

Good
Good
Good
Good

CRM

Salesforce.com
KDDI Business Outlook

Salesforce
KDDI

Good
Good

WebOS

StartForce

StartForce

Good

Mail

Bizmail (Ajax edition)

NTT Communications

Good

Static content
applications

CybozuBizmail (HTML edition)

CybozuNTT Communications

Good
Good

CRM: customer relationship management

3. Introduction of our methods
An overview of our methods is shown in Fig. 5.
These methods are algorithms that take as input the
web browser’s processing log data for networking,scripting, and rendering tasks; the output is the user
experienced wait time. This information makes it
possible to determine whether or not a decrease inperformance was caused by the terminal for an operation of any application. More specifically, our algorithms calculate a feature amount from the webbrowser’s networking, scripting, and rendering logs,
and output the estimated results by comparing theVol. 11 No. 5 May 2013(*) More than 80%: Good
Less than 80%: Unsatisfactory

conditions of the expected user wait time feature
amount pattern and the quality deterioration causedby the terminal feature amount pattern, which areprepared in advance.
The applications that were evaluated are listed inTable 1. As indicated, our methods can be applied towidely used applications such as Salesforce andMicrosoft Office Web Apps. To use our estimation
methods, it is necessary to install the browser plug-in
for the terminal targeted for estimation. This installation requires the user’s permission, so we applied themethods first to corporate applications.

Feature Articles

4. Summary and future work
With the development of in-browser processing
technology such as Ajax, the terminal processing
time has become a key factor of the wait time thatusers experience. This has caused a gap between traditional performance indicators such as HTTP
response time and API response time and the experienced wait time. We developed two methods forbrowser-based applications with dynamic content inorder to deal with this problem. One method is used
to estimate the wait time experienced by the user andHiroshi Yamamoto

Reference[1]

H. Yamamoto, S. Nakamura, H. Honda, D. Ikegami, and A. Takahashi,“The Consideration of Web Browsing Application Performance Factor,” IEICE-CQ2012-20, pp. 17–22, 2012 (in Japanese).

Hirotada Honda

Senior Research Engineer, IP Service Network
Engineering Group, NTT Network Technology
Laboratories.
He received the B.S. and M.S. degrees in information and computer science from Waseda University, Tokyo, in 1999 and 2001, respectively.
He joined NTT Service Integration Laboratories(now NTT Network Technology Laboratories) in2001. He has been working on the architecture
and performance evaluation of IP networks andweb applications. He is a member of the Instituteof Electronics, Information and Communication
Engineers (IEICE).

Research Engineer, IP Service Network Engineering Group, NTT Network Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees
in science from Keio University, Kanagawa, in
2000, 2002, and 2011, respectively. He joined
NTT in 2002. He is currently investigating the
playback quality estimation of progressive download-based video services. He is a member of
IEICE.

Sorami Nakamura

Akira Takahashi

Research Engineer, IP Service Network Engineering Group, NTT Network Technology Laboratories.
She received the B.S. degree in architecture
and building engineering and the M.S. degree inmathematical and computing sciences from
Tokyo Institute of Technology in 2008 and 2010,
respectively. Since joining NTT in 2011, she has
been working on quality design and managementin networks. She is a member of IEICE and the
Operation Research Society of Japan.

the other to determine whether or not a decrease inperformance was caused by the terminal.
In the future, we plan to develop an estimation
method that does not require a browser plug-in inorder to extend our methods to mass users.

Manager of the IP Service Network Engineering Group, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the B.S. degree in mathematics
from Hokkaido University in 1988, the M.S.
degree in electrical engineering from California
Institute of Technology, USA, in 1993, and the
Ph.D. degree in engineering from the Universityof Tsukuba, Ibaraki, in 2007. He joined NTT in1988 and has been engaged in the quality assessment of audio and visual communications. He
was a co-Rapporteur of ITU-T Question 13/12 on
Multimedia QoE and its assessment during the
2004–2008 Study Period. He is a Vice-Chairmanof ITU-T Study Group 12 (SG12) for the 2009–2012 and 2013–2016 Study Periods. He is aVice-Chairman of the Technical Committee ofCommunication Quality in IEICE. He received
the Telecommunication Technology CommitteeAward in Japan in 2004 and the ITU-AJ Award inJapan in 2005. He also received the Best Tutorial
Paper Award from IEICE in Japan in 2006 andthe Telecommunications Advancement Foundation Award in Japan in 2007 and 2008.

NTT Technical Review

Feature Articles: QoE Estimation Technologies

QoE Assessment Methodologies for
3D Video Services
Kazuhisa Yamagishi, Taichi Kawano, andKimiko Kawashima
Abstract

Technological advances in three-dimensional (3D) video enable users to watch 3D video content.
However, watching such content may cause visual discomfort and fatigue. This article gives an outlineof the relationship between 3D video degradation factors and quality of experience (QoE) and introduces QoE assessment methodologies and standardization activities.

1. Introduction
Users can now easily enjoy watching three-dimensional (3D) content on terminals such as 3D televisions (TVs), personal computers, and smartphones
due to the recent advances made in these terminals.
Video quality has also been improved by the introduction of high definition (HD). In addition, by introducing depth perception to video, 3D video gives a
new experience to users. However, some users complain of visual discomfort and fatigue from watching
3D video. To provide high-quality 3D video content,
it is important to design and manage services basedon the quality of experience (QoE), and to do this, a
3D video QoE assessment methodology is essential.
2. 3D video QoE
This section describes QoE for 3D video services.
The International Telecommunication Union, Telecommunication standardization section (ITU-T) Recommendation BT.2021 defines 3D video QoE interms of visual quality, depth perception, and visual
discomfort [1], [2]. As described in section 1, fatigue[3] from 3D video content is also an important factor.
In this article, we define QoE in terms of visual quality, depth perception, discomfort, and fatigue. TheseQoE factors are affected by the 3D video processing
chain, as shown in Fig. 1.
There are differences between the processing chain
and the human vision system (HVS) in 3D video

acquisition and display. For example, the position andangle of a camera do not match those of the humaneyes. In addition, when a user views 3D video content, they see an image formed from two video
images viewed separately by the left and right eye
through stereoscopic glasses in the rendering phaseof the processing chain. As a result, puppet theater*1,
cardboard *2, and spatio-temporal asynchronous
effects between the left and right views occur. Crosstalk*3 due to the stereoscopic glasses also occurs.
3D video is downsized and/or encoded in order toreduce the network bandwidth and the amount ofstorage needed. To use the existing infrastructure forcodec and transmission, the spatial resolution of theleft and right views, which are arranged in a side-byside frame-compatible format, is usually down-converted by half in the horizontal direction to maintainthe spatial resolution of a full high-definition (HD)2D video sequence. The video is encoded by MPEG2 (Motion Picture Experts Group-2) or H.264/AVC(advanced video coding) and is transmitted to a user
terminal such as a set-top box. Finally, the side-by*1	 The puppet-theater effect makes a 3-D image look unnaturally
small compared with the target image; people appear to be miniaturized puppets.
*2 The cardboard effect makes 3-D images look layered, i.e., consisting of flat objects against a flat background, though an observer can grasp the situation in front of and behind the shooting
target.
*3 Crosstalk appears because of imperfect view separation when asmall proportion of one eye’s image is also perceptible by theother eye.

NTT Technical Review

Feature Articles

QoE factors
Video quality Depth quality

Typical degradation- Puppet theater effect- Cardboard effect- Asynchronous effectbetween left and rightviews

Discomfort

Fatigue

Typical degradation- Spatial resolution reduction- Blockiness- Blurring- Difference in quality between left
and right views- Frame freeze (with skipping/without skipping)

Typical degradation- Crosstalk- Camera position/orientation- Camera parameters- Format- Format conversion- Coding scheme- Transmission system- Transmission error- Display type- Glasses- Monitor size
- Illuminance- Viewing distance/angle

Acquisition Editing

Encoding Transmission Decoding

Rendering

Fig. 1. 3D video processing chain.

side format video is decoded and up-converted to two
full HD video signals for the left and right views.
Thus, users perceive degradations in quality due tothe reduced spatial resolution in addition to coding
artifacts such as block noise. To prevent the degradation due to the reduced spatial resolution, the use of
two full HD video signals for left and right views,
which is called the frame-sequential format, is ideal.
In this case, an H.264/MVC (multiview video coding) is often used, which involves an inter-view prediction technique that encodes the right-view videousing both videos for left and right views in order toreduce the bit rate for the right-view video. With thissystem, service providers often encode the right-view
video at a much lower bit rate than the left on thebasis of binocular suppression. The two 2D video
signals for the left and right views have full HD resolution, but they have an asymmetric quality in addition to coding artifacts such as block noise.
Encoded 3D video is packetized and transmitted
over a network such as an IP (Internet protocol) or
terrestrial network. Packet loss sometimes occurs in
networks. Block noise occurs if there is no packetloss concealment (PLC) technique applied in the user
terminal. In contrast, if a PLC technique is applied in
the user terminal, freezing artifacts will be introduced
when the PLC scheme of the receiver replaces theerroneous frames (either due to packet loss or error
propagation) with the previous error-free frame until
a decoded picture without errors has been received.
Vol. 11 No. 5 May 2013This type of artifact is also called freezing with skipping. The rebuffering artifacts come from rebuffering
events at the receiver, which could be the result of a
stream arriving late. This type of artifact is also calledfreezing without skipping.
Degradation perceptions may change due to thedisplay size, room illuminance, viewing distance, andangle.
Therefore, as mentioned previously, because 3D
video QoE is affected by many factors, methodologies are needed to assess 3D video QoE.
3. 3D video subjective assessment methodology
Subjective assessment, in which users subjectively
evaluate 3D video QoE, is a fundamental quality
assessment technique. ITU-R Recommendation
BT.2021 was standardized for the 3D video quality
subjective assessment method. As described in section 2, it is important to develop methodologies thatassess depth perception, discomfort, and fatigue since
3D video QoE is affected by these factors, in addition
to visual quality. The Video Quality Experts Group(VQEG) is currently investigating subjective assessment methodologies concerning depth perception
and discomfort. However, it is often difficult to evaluate discomfort and fatigue using questionnaire-based
subjective assessment because the levels of these
indicators are sometimes low in questionnaires.
Therefore, it is important to supplement informationFeature Articles

54
Fatigueobtained from questionnaires with biological information such as heart rate, breathing rate, pupilchanges, and eye-blink responses when assessing
discomfort and fatigue. Our group has investigated
the relationship between fatigue and biological information. The relationship between fatigue and the difference in video quality between left and right viewsis shown in Fig. 2. In this figure, the lower the number
on the fatigue axis, the greater the amount of fatigue;
i.e., a value of 1 represents high fatigue, whereas 5
represents low fatigue. The relationship between theeye-blink rate and the difference in video quality isshown in Fig. 3. As shown in Fig. 2, fatigue scoreincreases as the difference in video quality increases.
As shown in Fig. 3, the eye-blink rate increases as thedifference in video quality increases. These results
suggest that fatigue can be evaluated using the eyeblink rate.

2

1
Small

dMOS-LR (= ABS(MOS-L – MOS-R)).
We compared the performance of our model with


Large

Difference in quality between left
and right views

Fig. 2. Difference in quality between left and right views
vs. fatigue.

4. 3D video quality objective
estimation methodology

15
Eye-blink rate (times/minute)

Developing an objective quality estimation modelthat can be used to estimate QoE using informationsuch as 3D video signals is essential for monitoring
QoE.
It is important to take into account the degradation
factors described in section 2 in order to develop sucha model. In principle, block noise, blurring, andfreezing due to encoding and transmission also occur
in 3D video services. Therefore, the 2D video quality
objective estimation model can be applied to 3Dvideo quality estimation. However, since quality degradation factors such as the difference in video quality between left and right views, the asynchronous
effect between left and right views, and crosstalk do
not occur in 2D video services, these factors need tobe taken into account in 3D video quality estimation.
Our group has been developing an objective quality
estimation model that takes 2D video quality for left
and right views, which is derived from a 2D video
quality objective estimation model, as input. Video
quality is denoted as a mean opinion score (MOS),
where 2D video quality for the left view is denoted as
MOS-L, 2D video quality for the right view is denoted as MOS-R, and the difference in video quality forleft and right views is denoted as

3

14
13
12
11
10
Small

Large

Difference in quality between left
and right views

Fig. 3. Difference in quality between left and right views
vs. eye-blink rate.

that of a conventional model used to calculate theaverage 2D video quality for left and right views.
Table 1 lists the performance values of our model andthe conventional model, i.e., the root mean squareerrors in the range of0 ≤ dMOS-LR ≤ 1 and 1 < dMOS-LR.
The results show that our model can estimate 3Dvideo quality in the range of
1 < dMOS-LR,
NTT Technical Review

Feature Articles

more accurately than the conventional model can.
VQEG is also investigating a 3D video quality
objective estimation model and discussing a test planthat will be used to verify the validity of such future
models.

Table 1. Quality estimation accuracy.
Proposed model

Conventional model0 ≤ dMOS-LR ≤ 1

0.22

0.24

1 < dMOS-LR

0.23

0.35

clarify factors that affect QoE and to develop a model
to estimate QoE in order to provide high-QoE 3D
video. Our group has been investigating subjective
assessment methodologies for 3D video quality andfatigue as well as an objective quality estimation
model. We plan to propose our model to VQEG in the
future. We will also develop subjective and objective
quality assessment methods for QoE other than thatfor 3D video quality using biological information. We
will promote the practical use of these methods,
which will contribute to providing a safe and pleasant
3D video streaming service.
References[1]

5. Conclusion
The introduction of 3D video has enabled service
providers to provide a new visual experience, e.g.,
depth perception, to users. However, some users havecomplained of visual discomfort and fatigue fromwatching 3D video. Therefore, it is important toKazuhisa Yamagishi

Research Engineer, Service Assessment Group,
Communication Traffic & Service Quality Project, NTT Network Technology Laboratories.
He received the B.E. degree in electrical engineering from Tokyo University of Science andthe M.E. degree in electronics, information, andcommunication engineering from Waseda University, Tokyo, in 2001 and 2003, respectively.
He joined NTT in 2003. He has been engaged insubjective quality assessment of multimedia
telecommunications and image coding. He iscurrently working on quality assessment of multimedia services over IP networks. He has been
contributing to ITU-T SG12 since 2006. From2010 to 2011, he was a visiting researcher at
Arizona State University. He received the Young
Investigator’s Award from the Institute of Electronics, Information and Communication Engineers (IEICE) in 2007 and the Telecommunication Advancement Foundation Award in Japan in
2008. He is a member of IEICE.

[2]

[3]

ITU-R Recommendation BT.2021, “Subjective methods for theassessment of stereoscopic 3DTV systems,” Aug. 2012.
M. Lambooij, W. Ijsselsteijn, M. Fortuin, and I. Heynderickx, “Visual
discomfort and visual fatigue of stereoscopic displays: a review,”
Journal of Imaging Science and Technology, Vol. 53, No. 3, pp.
030201–14, 2009.
K. Yamagishi, T. Kawano, and T. Hayashi, “Effect of Difference in 2D
Video Quality for Left and Right Views on Overall 3D Video Quality,”
Proc. of 2012 IEEE International Conference on Image Processing(ICIP 2012), Orlando, Florida, USA, 2012.

Kimiko Kawashima

Researcher, Service Assessment Group, Communication Traffic & Service Quality Project,
NTT Network Technology Laboratories.
She received the B.E. and M.E. degrees inengineering from Keio University, Kanagawa, in2008 and 2010, respectively. Since joining NTT
in 2010, she has been engaged in research on
quality assessment of visual communication services. She is currently working on quality assessment of 3D services. She is a member of IEICE.

Taichi Kawano

Researcher, Service Assessment Group, Communication Traffic & Service Quality Project,
NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in engineering from Tsukuba University, Ibaraki, in2006 and 2008, respectively. Since joining NTT
in 2008, he has been engaged in research on
2D/3D video quality assessment. He received theYoung Investigator’s Award from IEICE in 2011.
He is a member of IEICE.

Vol. 11 No. 5 May 2013



Regular Articles

Historical Overview of
Semiconductor Device Reliability for
Telecommunication Networks––
Field Data, Prediction Model of Device Failure
Rate, and Wear-out Failure Analyses at NTT
Noboru Shiono, Eisuke Arai, and Shin’ichiro Mutoh
Abstract

The 1960s brought about the advent of the intense development of semiconductor devices for industrial
and commercial applications. Since then, NTT has taken the initiative in introducing semiconductor
devices into telecommunication networks to realize highly reliable and miniaturized equipment withhigh performance.
This article reviews the failure physics approach we have been using to develop highly reliable
semiconductor devices. It also summarizes our findings regarding device failures in field use anddiscusses a failure rate prediction model based on the findings.

1. Introduction
From the 1960s to the 1980s, NTT’s use of semiconductor devices was the primary factor in the successful development of highly reliable and miniaturized high-performance telecommunications equipment. In the 1960s, the semiconductor industry wasin its infancy. Therefore, it was not really known how
reliable semiconductor devices would be, and experience in using them in telecommunications equipment
was lacking. At that time, NTT was the first company
in the world to introduce transistors into telecommunications equipment [1]–[7].
In the early 1960s, NTT introduced a failure physics approach (also known as the physics of failure),
described in the next section, in order to develop
highly reliable semiconductor devices for telecommunications equipment. The target values for device
reliability required in network system design were alifetime longer than 25 years and a failure rate lower
than 0.2–150 FIT (failure in time, which is defined as
the number of failures per billion device hours
(10-9/h)), depending on the type of device (fromdiodes to LSIs (large-scale integrated circuits)) andthe kind of equipment [4]. For semiconductor devices
that had been developed at that time, accelerated life
tests (longer than 10,000 hours) and long-term operation tests at room temperature using a large numberof devices were conducted to confirm that those targets had been achieved. In addition, field failure tests
were carried out with actual working equipment toconfirm the high reliability of semiconductor devices
and of passive and mechanical components as well.
On the basis of the results of those field failure
tests, we developed a failure rate prediction model forelectronic components including semiconductor
devices to use in the reliability design of the nextstage of our telecommunications equipment [8], [9].
The objectives of this failure rate prediction model
were the same as those of the well-known MILHDBK-217 (Military Handbook) model for reliability prediction of electronic equipment [10], and theprocedures were similar.
The development of reliability assurance standards
NTT Technical Review

Regular Articles

New telecommunications system planning(Target value of system life and failure rate isdetermined.)

Mass production

Setting of lifetime and failure rates forsemiconductor devices

No good
Qualification testing ofC
mass-produced devices

Selection of appropriate design and fabrication
technology (reliability design)

Go
Screening

Design and fabrication of reliability test
structures

No good
D Lot-acceptance testing
No goodA

Accelerated life testingof test structures

Go
Shipment

Go
Trial fabrication of real devices

B

Reliability testingof real devices

No good

Go
Criteria for each testing point:A- Failure mode extraction
using test structures- Whether the estimatedfailure rate is lower than
the target value or not

B

C

D

- Whether degradation modes
are a matter or not- Whether lifetime andfailure rate of wear-out
failures are lower than thetarget values or not- Whether lifetime andfailure rate are lower than
the target values for massproduced devices or not- Whether the produced lot
is normal or not

Fig. 1. Procedure for developing highly reliable semiconductor devices for telecommunications equipment.

such as MIL-STD (Military Standard) -19500 [11]and MIL-STD-38510 [12] started in the late 1950s,
and NTT applied them to semiconductor devices
installed in NTT equipment to ensure their high reliability [8]. Over the years, NTT also developed andperiodically revised its own assurance standards toreflect advances in device technology. These standards were finally withdrawn in 1987 because thereliability of semiconductor devices had been
improved to the level where such standards were nolonger necessary.
2. Development of highly reliable devices
based on the failure physics approach
The failure physics approach for developing highly
reliable devices is as follows; first, various failuremodes are investigated in accelerated life tests. Next,
Vol. 11 No. 5 May 2013

associated failure mechanisms are clarified, anddevice reliability is improved by optimizing device
structures and fabrication processes. Finally, a reliability assurance procedure is carried out; this procedure includes conducting wafer acceptance tests,
screening tests of encapsulated devices for eliminating early failures, and lot-acceptance tests.
The procedure we followed for developing highly
reliable semiconductor devices is shown in Fig. 1,
and examples of devices used in NTT equipment aresummarized in Table 1 [13], [14]. The relationshipbetween failure rate and working time is known as thebathtub curve, which depicts the early failure period,
random failure period, and wear-out failure period.
Early failures are caused by latent defects induced
during fabrication, and the failure rate decreases withtime (β < 1, where β is the shape parameter in a
Weibull distribution). Random failures are caused byRegular Articles

Table 1. Main devices used in various telecommunication systems, failure modes, and countermeasures for higher
reliability.
System (System name/year installed)

Main devices(type or process used)

Main failure modes

Countermeasures for higher
reliability

Coaxial cable transmission
system(CP-12MTr/1967)

Bipolar transistors(direct wire bonding on
metallization of E-B junction)- Wire bond breakage due to
Au-Al compound formation
between Au wire and Al
metallization pad on junction- E-B junction short due to Au
penetration of Au wire into
junction- Control thermal processduring wire bonding andcontrol bond strength- Change Au wire to Al wire

Electronic exchangesystem (D10/1972)

Logic ICs (bipolar CSL type)- Open failure of Al
metallization due tocorrosion- Improve passivation film

Electronic exchangesystem (D10-HCP/1977)

4K DRAM(n-channel MOS process)- Increase in leakage current- Decrease in holding time

- Control contamination- Optimize screening
conditions

Submarine coaxial cablesystem(CS-36M/1977)

Bipolar transistors (Au/Pt/Ti
electrode, BeO substrate forhigh thermal conductivity)- Metal penetration into
junction- hFE degradation- Improve electrode metal
structure from Al to Ti/Pt/Au
electrode- Improve passivation film
from single layer of SiO2 to
SiO2 /SiN- Use individual device
assurance method

Digital electronic exchangesystem(D70/1982)

High-voltage bipolar LSIs
(dielectric isolation, pnjunction isolation, plastic
encapsulation)- Open failure of Al
metallization due tocorrosion- Degradation of breakdown
voltage for isolation- Adopt SiN passivation film
- Adopt EPIC dielectricisolation structure combinedwith field plate

64K DRAM- Soft error- Use chip coating with resin- Adopt ECC technique

Optical fiber transmission
system(F-400M/1983)

InGaAsP/InP BH-LDs(Fabry-Perot type, buriedheterostructure)- Increase in thermal
resistance in die bonding tosubstrate- Short failure due to Sn
whisker growth around die
bonding- Degradation of opticalpower output- Improve metallized layer andadopt silicon heat sink
- Use Au-rich AuSn die
bonding- Use individual device
assurance method

ISDN network(ISDN/1988)

Submicrometer custom LSIs
(CMOS, BiCMOS)- Open failure due toelectromigration and stressmigration of Al metallization- Vth degradation due to hotcarrier effect- TDDB of gate oxide- Use multilayer metallization
structure (Al-Si-Cu/Ti/TiN/Ti,
etc.)- Control process of H2O
content in interlayer dielectric
film- Use high-voltage and lowtemperature screening

BeO: beryllium oxide
BH: buried heterostructure
CSL: controlled saturation logic
Cu: copper
E-B: emitter-base
ECC: error-correcting code
EPIC: epitaxial passivated integrated circuit
H2O: hydrogen dioxide (water)

InGaAsP: indium gallium arsenide phosphide
ISDN: integrated services digital network
Pt: platinum
Si02: silicon dioxide
SiN: silicon nitride
Sn: tin
TDDB: time-dependent dielectric breakdown (gate oxide breakdown)

small latent defects that occur after relatively longterm operation at a constant failure rate (β = 1). Wear

out failures occur after long-term operation due towear-out and fatigue, and the failure rate tends to
NTT Technical Review

Regular Articles

Failure rate in field use (FIT)

Diodes100

Transistors
Bipolar ICs

10

1

0.1
1965

1970

1975

1980

Year installed

Fig. 2. Failure rate in field use vs. year installed in NTT equipment for semiconductor devices.

increase rapidly during this period (β > 1).
Countermeasures consisting of fabrication processcontrol and proper screening were effective in reducing the failure rate of early failures. For random failures, long-term tests conducted under actual working
conditions confirmed that failure rates were lower
than the target value. Wear-out failures, which should
never occur during the guaranteed lifetime of adevice, were prominent for miniaturized LSIs andoptical devices. These failures are classified into degradation-related failures and catastrophic ones.
Examples of degradation failures are a decrease in the
optical output power of laser diodes (LDs) or changes
in the current gain (hFE) of bipolar transistors during
operation. In a circuit affected by such degradation,
the extent of the degradation has to be accurately
estimated in order to analyze its effect on circuit
operation in the intended working conditions. Examples of catastrophic failures are the lifting of Au(gold) wire bonding and stressmigration in Al (aluminium) interconnections in plastic-encapsulated
LSIs. Wear-out failure analyses that we performed forplastic encapsulated LSIs and LDs are described in
more detail in section 4.
3. Failure rate in field use andreliability prediction model
3.1 Failure rate data in field use
From the 1960s to the 1980s, NTT collected a large
amount of data on field failures of devices in NTT
telecommunications equipment. The devices wereVol. 11 No. 5 May 2013

typically operated in an air-conditioned room with amaximum junction temperature of 55ºC (ambient
temperature 40ºC) and relative humidity of 20–70%[5], [15]–[25].
The relationships between the failure rate of devices(in failures/109 hours = FIT) in field use and theinstallation year for diodes, bipolar transistors, andbipolar integrated circuits (ICs) are shown in Fig. 2.
These values were selected from field data of morethan 1×10 10 device·hours for diodes, 4×10 9
device·hours for bipolar transistors, and 5×10 8
device·hours for bipolar ICs, which corresponded to
one to three years of operation. The lower limits ofthe failure rate estimation are 0.01 FIT for diodes,
0.25 FIT for transistors, and 2 FIT for bipolar ICs.
These lower limits are one order lower than theobserved failure rate as described below.
As shown in Fig. 2, the failure rate decreased withtime and approached a certain constant value depending on the device group. These curves are similar tothe so-called learning curve, and the constant value
corresponds to the failure rate in the random failure
period for mature devices. The failure rate seemed to
become higher as the process complexity increased.
The failure rate in the random failure period wasestimated to be about 0.2 FIT for diodes, about 2 FIT
for transistors, and about 5 FIT for bipolar ICs. The
failure rate observed in field use from the 1960s to the
1980s for more precisely divided device groups in the
random failure period is plotted in Fig. 3. The figure
shows that somewhat high failure rates were observed
for metal-oxide semiconductor (MOS) ICs, whichRegular Articles

Failure rate in field use (FIT)

100

10

1

EC
LI
C
EC
LL
SI
an
al
og
IC
C
M
O
SI
C
C
M
O
SL
SI
D
R
AM
(1
K)
D
R
AM
(4
D
K
R
AM )
(1
6K
)

LI
C
TT

Tr

D
i

0.1

Di: diode
Tr: transistor
TTL: transistor-transistor logic
IC: integrated circuit

ECL: emitter-coupled logic
CMOS: complementary MOS
DRAM: dynamic random access memory

Fig. 3. Failure rate in field use in random failure period for various semiconductor devices.

had just been developed and had not yet matured inthose days, though the failure rates were less than 100
FIT. We also examined the failure rate for LSIs developed in the 1990s by conducting accelerated life tests
with a large number of LSIs for about 3000 hours. We
found that the failure rates of memory LSIs such as
64-K–1-M DRAM (dynamic random access memory) were below 100 FIT, and those of CMOS (complementary MOS) logic LSIs (about 150-K gates)
fabricated with 0.25–0.5 µm technology were below
200 FIT. The failure rates of these LSIs were expected
to decrease in the accelerated life tests and approacha constant value below 100 FIT if the tests were continued beyond 3000 hours, because no failures wereobserved within the first 3000 hours.
3.2 Failure rate prediction models
3.2.1 NTT model
A failure rate prediction model was developed on
the basis of the collected field data and used in thenext stage of equipment design [8], [9], [26], [27].
The NTT failure rate prediction model (called the
NTT model hereafter) was sought to determine theconstant failure rates in the random failure period andwas similar to the well-known MIL-HDBK-217
model. It was established for semiconductor devices
and passive and mechanical components used forNTT equipment. The failure rate of ICs is expressedas
λ = λb · πQ · (πE + πT · πV),

(1)where λ is the predicted failure rate, λb is the basic
failure rate (FIT), πQ is the quality factor, πE is theenvironmental factor, πT is the temperature factor,
and πV is the power supply voltage factor.
The basic failure rate λb was defined for eachdevice type, integration level, and technology. For
example, the λb of bipolar digital logic ICs and MOS
DRAM is listed in Table 2.
Quality factor πQ was given a value from 1 to 4,
depending on the quality grade defined by NTT; it
was 1 for highly reliable ICs (namely, quality-controlled devices used in NTT equipment). Environmental factor πE was given a value from 0.3 to 1.8,
depending on the operating environment such as an
air-conditioned room, vehicle, or mobile telephone; it
was 0.3 for an air-conditioned room. Power supply
voltage factor πV was expressed as πV = 0.35 × exp(0.21VDD) for CMOS ICs and 1 for other devices.
Temperature factor πT is given as
πT = exp0.3  1 - 1  + exp 0.7  1 - 1  , (2) k 339 273+Tj  k  356 273+Tj 
NTT Technical Review

Regular Articles

Table 2. Example of basic failure rate of ICs.
λb (FIT)

Category
Bipolar digital logic ICs
MOS DRAM

5.0
3.81B

for
0.25

for

Q<
= 100
< 16 K
B=

0.50Q0.50

for

0.50

for

0.337B

Q>= 100> 16 K
B=

Q: number of active transistors
B: bits; K = 1024

where k is Boltzmann’s constant and Tj is junction
temperature at the operating condition. This formula
was derived by considering the contribution to thefailure of two activation energies (Ea) of 0.3 and 0.7
eV. Here, various failure mechanisms showed different activation energies, and the major failure mechanisms for MOS ICs were gate oxide breakdown(TDDB) with Ea=0.3 eV and electromigration or
stressmigration of metallization with Ea=0.7 eV.
Activation energies for these major failure mechanisms were therefore introduced to define the temperature factor in the NTT model.
Unlike the MIL-HDBK-217 model, the NTT model
considered the effect of packaging only for hermetic
or non-hermetic (plastic-encapsulated) sealing anddid not take the effect of package type or size into
account.
3.2.2 C omparison of NTT model with otherfailure rate prediction models
The final revised version of the NTT model wascompleted in 1981 and published in 1982 [8]. In thosedays, comparable prediction models for electronic
components were MIL-HDBK-217 (version C (1979)and version D (1982)), and Bellcore TR-332 [28].
Later, other failure rate prediction models weredeveloped such as PRISM [29] and IEC/TR (International Electrotechnical Commission/Technical
Report) 62380 [30]. These models consider not only
individual components but also the total system
operation, and they differ from device-oriented models such as the NTT model and MIL-HDBK-217.
Quantitative comparisons of the predicted trend of
failure rates among these various models were needed
but had not been carried out [31], [32]. We therefore
performed comparisons between six device-oriented
models of the NTT model (1981), three versions of
the MIL-HDBK-217 model (C(1979), E(1986),
F(1991)), and two versions of the Bellcore model(TR-332 issue (1997), SR-332 issue 2 (2006)). (Notethat TR-332 was changed to SR-332 in 2001.) Operating conditions were assumed to be Tj = 55ºC (ambient temperature of 40ºC) and an air-conditioned
Vol. 11 No. 5 May 2013

room, which corresponded to “Ground, Benign” in
MIL and also to the NTT electronic switching system
environment. The quality grade was selected as thehighest reliable level at NTT, i.e., πQ = 1, which was
equivalent to MIL-class B. The power supply voltage
factor was a standard voltage, and πV = 1. Under these
conditions, comparisons of failure rate were performed for NMOS (n-channel MOS) DRAM andbipolar digital logic ICs as a function of the scale ofintegration. The results are shown in Figs. 4 and 5.
For DRAM, the failure rate prediction curve
obtained with the NTT model lies between MILHDBK-217 C(1979) and 217 E(1986), and the relationship between failure rate and DRAM size seems
to shift in parallel with subsequent versions, i.e., theyear of publication. For digital logic ICs, the prediction curves tend to converge at around 200 FIT withincreases in the number of gates, regardless of themodel. The features of the relationship between failure rate and scale for DRAM are somewhat different
from those for bipolar logic ICs. This difference canbe explained as follows; the process technology for
DRAM had been revised with the very large scale
production more quickly than it had for bipolar logic
ICs. Consequently, the yield and reliability were
improved more quickly compared with bipolar digital
ICs. The trend in the predicted results indicated thatthe failure rate of LSIs would converge to a constantvalue regardless of the scale of integration, and thatvalue was estimated to be 100–200 FIT even for the
maximum scale of integration for a given year. On thewhole, the relationship between failure rate and scaleof integration can be assumed to shift in parallel withthe fabrication year due to advances in technology.
Although these prediction models involving the NTT
model are old, they can still be used as references forpredicting failure rates of recently developed
devices.
4. Examples of analyses forwear-out failure modes
Wear-out failure is divided into catastrophic failureRegular Articles

217 C(1979)NTT (1981)
TR-332 issue 6(1997)

217 E(1986)

Failure rate (FIT)

100

10

SR-332 issue 2(2006)
217 F(1991)
Tj = 55°C

1
1K

10 K

100 K

1M

10 M

100 M

1G

10 G

DRAM size (bits)

Fig. 4. Predicted failure rate vs. size of DRAM for various failure rate prediction models.

SR 332 issue 2(2006)
TR 332 issue 6(1997)

Failure rate (FIT)

100

NTT
(1981)
NTT(1981)

10217 F(1991)
217 E(1986)

217 C(1979)

Tj = 55°C

1
10

102

103

104

105

Bipolar Digital Logic Array (gates)

Fig. 5. Predicted failure rate vs. number of gates in bipolar logic IC gate arrays for various models.

and degradation failure modes, as mentioned previously. The time to reach wear-out failures has tended
to decrease with the rapid progress in the miniaturization of device dimensions used in LSIs since the latter
half of the 1980s. Reliability analyses of plasticencapsulated LSIs and LDs are described below asrespective examples of catastrophic and degradation
failures.
4.1 Reliability analysis for catastrophic failures
A reliability analysis was carried out for plasticencapsulated MOS DRAMs, in which typical failure
NTT Technical Review

Regular Articles

gration of Al(Si) metallization for the 1-M DRAM.
Au wire-bond failures were induced by thermal
expansion mismatch among the molding compounds(10–20 ppm/ºC), Au wire (14 ppm/ºC), and Si(3 ppm/ºC), and also by decreases in the thermal
expansion coefficient of molding compounds duringhigh-temperature aging. Stressmigration failure was

5

Cumulative failures F (%)

99.9
99
95
90
80

6

7

ln(t)
8

9

10

11

2
1
0

50

64 K
150°C

30
20
15
10

–2

1M
175°C

5
1M
175°C

64 K
175°C

256 K
175°C

–3
–4

1
100

–1

1000

Cumulative failures lnln(1/(1-F(t)))

modes were open failures caused by the lifting of Au
wire-bonds and stressmigration of Al(Si) metallization [13]. Weibull plots of failures observed in hightemperature operating tests for 64-K, 256-K, and 1-M
DRAMs are shown in Fig. 6. Failure analyses clarified that the failures involved the lifting of Au wirebonds for the 64-K and 256-K DRAM and stressmi-–5100,000

10,000Time (h)(a) Weibull failure distribution

100

Failure rate (FIT)

1-M DRAM

256-K DRAM

1064-K DRAM

1

0.1
0.1

1

10

102

103

104

105

106

Time (years)(b) Failure rate vs. time in use

Fig. 6. (a) Failures observed for plastic encapsulated DRAM in high-temperature operation tests and (b) estimated failure
rate vs. time under normal operating temperature (Tj = 55ºC).

Vol. 11 No. 5 May 2013



Regular Articles

Table 3. Weibull parameters determined from high-temperature operation tests for NMOS DRAM.
Deviceη (scale parameter)at 175°C (h)β(shape parameter)

Activationenergy (eV)64-K DRAM

3400

6.5

256-K DRAM

5200

6.5

1-M DRAM

2300

2.5

1.25
1.25
0.55

caused by tensile stress due to the thermal expansion
difference between the Al interconnection and theintermediate layer.
Weibull parameters determined from the results of
the accelerated life tests shown in Fig. 6(a) are listed
in Table 3. The values of the shape parameter β were
6.5 for 64-K and 256-K DRAM and 2.5 for 1-M
DRAM, which indicates the wear-out failures. The
activation energy of the Au wire-bond open failure
was estimated to be 1.25 eV for 64-K DRAM, andwas assumed to be the same for the 256-K DRAM.
The activation energy of the Al(Si) stressmigration
failure observed in the 1-M DRAM in the 150ºC and175ºC tests was estimated to be 0.15 eV. However, thefailure rate for stressmigration was known to have aplateau around 180ºC and to show complicated temperature dependence [33]. Therefore, 0.55 eV wasadopted as the activation energy for failure rate estimation, in which the energy was derived from testing
below 150ºC using other test structures [34].
The relationship between estimated failure rate andoperating time at the operating temperature of 55ºC
derived from the above parameters is shown in Fig.
6(b) in a Weibull distribution. Here, the failure rate in
the Weibull distribution was obtained byβ t β−1
λ(t) = (η
) (η)



(3)

The target value of the failure rate for catastrophic
wear-out failures was set at 10 FIT after 25 years. The
failure rates of 64-K and 256-K DRAM were far
below the target value, while that of the 1-M DRAM
reached the target value after only one year, as shownin Fig. 6(b), which indicates an obvious need for
improvement. The1-M DRAM was subsequently
improved by using Al(Si, Cu) interconnections
instead of Al(Si) interconnections, and the lifetime of
the improved 1-M DRAM was confirmed to be 100
times longer than the former one.
Further, room-temperature operating tests werecarried out over 17,400 hours with 320 samples, andthe results confirmed that the rates of random failuresof 64-K DRAM were below 165 FIT. For the 1-M
Failure mode
(open failure)
Lifting of Au wire-bonds
Lifting of Au wire-bonds
Stressmigration of Al(Si) interconnection

DRAM, 150ºC operating tests using 110 samples fora duration of 10,000 hours confirmed that the rates ofrandom failures were below 62 FIT. The analyses ofwear-out failures and random failures confirmed thatthe total failure rates satisfied the target values.
4.2 Reliability analysis for degradation failures
Degradation of electrical characteristics is a major
concern for semiconductor devices such as LDs,
whose optical output power degrades with time. Degradation analyses, therefore, are needed in addition to
random failure examination and are typically obtainedby conducting accelerated life tests. It is imperative to
confirm that the amount of degradation during theguaranteed duration of use is within the designatedvalue by analyzing the degradation characteristics. A
reliability analysis was done to determine the characteristic degradation of LDs used in the FS-400M
submarine optical telecommunications system.
Samples were Fabry-Perot buried type InGaAsPLDs with a 1.3 μm wavelength. In actual applications,
an automatic power control (APC) circuit was used tomaintain a constant optical output power; i.e., degradation of the constant output power of LDs was maintained by the increase in driving current in the APC
circuit. The failure criterion of the LD degradation
was normally defined by a 50% increase in driving
current. The accelerated life tests were conducted in
conditions of 50ºC and 70ºC temperatures with an
output power of 5 mW, and also at 10ºC and 5 mW,
which was the actual operating condition. In total,
1000 samples were tested, with 100 samples used forthe 10ºC test, 800 samples for the 50ºC test, and 100
samples for the 70ºC test. The rate of random failures
was also estimated by testing the same 1000 samples[35].
4.2.1 D egradation mode of electrical characteristics
The degradation mode was a decrease in optical
output power, and the degradation corresponded to anincrease in the driving current in the APC circuit,which could be easily monitored. The typical
NTT Technical Review

{

}



(4)

The estimation of failure rate versus time (years) at
10ºC in the actual undersea environment is shown in
Fig. 8(b). From this plot, a failure rate of 220 FIT
after 25 years was estimated, while 0.001 FIT after 25
years was estimated when 0.8-eV activation energy
was applied.
4.2.2 Random failure modeLong-term reliability testing for 19,000 hours at
maximum was carried out for 1000 samples as
described above. Except for optical output degradation, no catastrophic failures were observed in thesetests. A failure rate lower than 13 FIT (CL=60%) was
estimated from these tests of 7.3 × 107 device·hours
at 10ºC, where 0.35-eV activation energy was used.
Thus, the total rate of wear-out failure and random
failure was estimated to be below 233 FIT, which was
below the target value of 300 FIT at 10ºC operationand 25 years duration.
4.2.3 Screening
The operating test results showed a relatively goodcorrelation between the increase in driving current at
1000 hours and the lifetime estimated using the procedure described above with a correlation coefficientof -0.62. This indicates that a failure rate one order ofVol. 11 No. 5 May 2013

10

1

1000
Time (h)10,000Fig. 7. D riving current degradation of LDs used in FS400M system during high-temperature operating
test.

99
Cumulative failures (%)−
t
1√2·exp - 2 ·(1n( µ ) )2σ.
λ(t) =−
1 ·1n( t)√π·t·σ·erfc( −
µ)
√2σ
2

70°C, 5-mW APC

0.1
100

70°C 5 mW
9050µ = 1.4 × 105 h
σ = 0.99

10
1
104

105

Lifetime (h)

106

107

(a) Log-normal failure distribution

400

Failure rate (FIT)

degradation characteristics in the 70ºC test are shownin Fig. 7. The driving current increased with time as
DId/Io (t) ∝tn, where DId is the increase in driving current, Io is the initial driving current, t is the test time,and n is a constant. The value of n varied with thesamples in the range of 0.4–1.0. The lifetime, which
is defined as the time to reach a 50% increase in driving current, was estimated by extrapolating the relation shown in Fig. 7 for each sample. A log-normal
plot of cumulative failures versus lifetime for the
70ºC operating test is shown in Fig. 8(a). This lognormal distribution was approximated by the medianlife of µ = 1.4 × 105 hours and the standard deviationof σ = 0.99, except for the long lifetime portion.
Although the activation energy was estimated to beabout 0.8 eV from the data at 50ºC and 70ºC, the lowest activation energy of 0.35 eV used in semiconductor devices was applied in order to obtain a more
severe prediction in the actual lifetime estimation.
The failure rate is given by the following equation forthe log-normal distribution [36]:Ratio of increase in driving current
to initial current ∆ Id / I0 (%)

Regular Articles

350

Tα =10°C300µ = 1.7 × 106 h

250

σ = 0.99

200
150
100
50
0

0

10

20
30
40
Time (years)(b) Failure rate vs. time (years)

50

Fig. 8. ( a) Lifetime distribution of LDs derived from Fig. 7and (b) failure rate prediction at 10ºC (failure
criterion is 50% increase in driving current).

10Regular Articles

magnitude lower than 300 FIT could be obtained by
selecting LDs with a smaller increase in driving current after 1000 hours of aging. This procedure was
implemented as the third step of screening, where thetotal screening consisted of three steps: 70ºC/150 mA
for 100 h, 70ºC/8 mW with APC for 100 h, and70ºC/5 mW with APC for 1000 h [35]. The first andsecond steps were used to avoid early failures andstabilize the electrical characteristics, and the third
step was done to select a long-lifetime device. This
screening method was called the individual device
assurance method because the electrical characteristics of each LD at all stages of production and screening were acquired individually, and because it was
expected that this method would make it possible toselect LDs with better stability and longer lifetime.
This individual device assurance method was also
applied for the selection of bipolar transistors used in
the submarine coaxial transmission system (CS36M), in which transistors with high stability, i.e.,
degradation of hFE of less than 3%, were required during the intended duration of service [37].
4.3 Future challenges
Recently, wear-out failure has become the mostsignificant issue in the reliability of advanced LSIs
because the process dimensions have been miniaturized to a size of about several tens of nanometers, andthin plastic encapsulation of high pin counts has been
applied, which results in shorter lifetimes. For opticalsemiconductor devices, the time to a wear-out failuresuch as the degradation of optical output power has
been the most important issue ever since the introduction of optical devices. For such devices, there is aneed for more precise reliability analyses and accurate failure rate estimation for both catastrophic failure and degradation modes.
In line with the recent ITRS (International Technology Roadmap for Semiconductors) [38], it has been
proposed that the failure rate for long-term reliability
should be determined according to system scale. For
example, 1000 FIT/chip is required for one chip/system, while 1 FIT/chip is needed for 1000 chips/system. However, 1 FIT/chip is not reasonable because itis very difficult or even impossible to ensure by reliability testing when considering the device·hours andtesting cost. The future challenge in ensuring highreliability of advanced devices is to provide the mostaccurate reliability information based on the advanced
failure physics approach to system designers, whowill then be able to design reliable systems by introducing a redundant unit such as multiple devices used
11

in parallel.
5. Summary
This article presented a review of the developmentof highly reliable semiconductor devices used in NTT
telecommunications equipment over a period of several decades. The findings obtained and the futurechallenges are summarized as follows:(1)	Highly reliable semiconductor devices weredeveloped through the failure physics approach
and were assured through accelerated life testing and field failure examination to have a lifetime longer than 25 years and failure rates from
0.1 to150 FIT (depending on device type),
which were adequate for application to NTT
telecommunications equipment.
(2)	Field failure data for semiconductor devices
were collected for various types of operating
equipment from the 1960s to 1980s. The data
revealed that the failure rate decreased with time
and tended to approach a constant value of lessthan about 100 FIT regardless of the device typeor scale of integration.
(3)	The failure rate prediction model, called the
NTT model, was developed based on field failure data collected and used in equipment reliability design. Comparisons between the NTT
model and other models used in those days, e.g.,
MIL-HDBK-217 and Bellcore TR-332, showedalmost the same prediction results.
(4)	Since wear-out failure modes vary with device
type or technology, it is necessary to clarify failure modes and failure mechanisms for relevant
devices by conducting accelerated life tests andfailure analyses, and to predict accurate failure
rates in actual-use conditions. The use of suchaccurate failure rate data will lead to the designof reliable equipment.
References
[1]
[2]
[3]
[4]

[5]

S. Shikama, “Investigations on Improvements and Life Testing Methods of Germanium Transistor,” NTT R&D, Vol. 10, No. 11, p. 2217,
1961 (in Japanese).
D. Kumagai, “Cable Carrier Transmission Systems Transfiguration––
Its Possibilities and Limitations,” NTT R&D, Vol. 17, No. 9, p. 1723,
1968 (in Japanese).
T. Kojima, “Research on Semiconductor Devices in the ElectricalCommunication Laboratory,” NTT R&D, Vol. 19, No. 10, p. 1809,
1970 (in Japanese).
K. Takahara and K. Hanawa, “Review of the Development of High
Reliability Techniques for Submarine Repeater Components,” NTT
R&D, Vol. 23, No. 3, p. 339, 1974 (in Japanese).
R. Kaneoya, “Research of Component Reliability in Electrical

NTT Technical Review

Regular Articles

Communication Laboratory, NTT R&D, Vol. 29, No. 7, p. 1193, 1980
(in Japanese).
[6] M. Watanabe, “Thirty Years in Semiconductor Research and Development––Trajectories and Tactics toward VLSI,” NTT R&D, Vol. 32,
No. 8, p. 1607, 1983 (in Japanese).
[7] H. Mukai, “R&D LSI Technology for Telecommunication in the Last
20 years,” NTT R&D, Vol. 39, No. 4, p. 491, 1990 (in Japanese).
[8] T. Mihara, T. Sugimura, N. Mori, and K. Hiraoka, “Reliability Evaluation of Semiconductor Devices in NTT,” NTT Facilities, Vol. 34, No.
6, p. 48, 1982 (in Japanese).
[9] Nippon Telegraph and Telephone Corporation, “Standard Reliability
Table for Semiconductor Devices,” Mar. 1985.
[10] MIL-HDBK-217, “Reliability Prediction of Electronic Equipment,”
Version C (1979), Version E (1986), Version F (1991), Version F(Notice 2) (1995).
[11] MIL-STD-19500, “Military Specification, Semiconductor Devices,
General Specification For,” 1994.
[12] MIL-STD-38510, “Military Specification, Microcircuits, GeneralSpecification For,” 1991.
[13]–[27] NTT Technical Review Reference Site (in Japanese).
[13] http://www.ntt.co.jp/milab/doc/201305d001.pdf[14] http://www.ntt.co.jp/milab/doc/201305d002.pdf[15] http://www.ntt.co.jp/milab/doc/201305d011.pdf[16] http://www.ntt.co.jp/milab/doc/201305d013.pdf[17] http://www.ntt.co.jp/milab/doc/201305d014.pdf[18] http://www.ntt.co.jp/milab/doc/201305d005.pdf[19] http://www.ntt.co.jp/milab/doc/201305d017.pdf[20] http://www.ntt.co.jp/milab/doc/201305d016.pdf[21] http://www.ntt.co.jp/milab/doc/201305d015.pdf[22] http://www.ntt.co.jp/milab/doc/201305d018.pdf[23] http://www.ntt.co.jp/milab/doc/201305d012.pdf[24] http://www.ntt.co.jp/milab/doc/201305d006.pdf

Noboru Shiono

Senior Manager, Reliability Center for Electronic Components of Japan.
He received the M.E. and Dr.Eng. degrees fromTohoku University, Miyagi, in 1971 and 1987,
respectively. He joined the Nippon Telegraph andTelephone Public Corporation (now NTT) Electrical Communication Laboratories in 1971 andwas engaged in R&D of highly reliable semiconductor devices. He moved to the Reliability
Center for Electronic Components of Japan,
where he has been researching the quality andreliability of electronic components and electron
devices.

Eisuke Arai

Emeritus Professor, Nagoya Institute of Technology.
He received the B.E. and Dr.Eng.degrees from
Nagoya University, Aichi, in 1965 and 1975,
respectively. In 1965, he joined the Nippon Telegraph and Telephone Public Corporation (nowNTT) Electrical Communication Laboratories,
where he was engaged in R&D of LSI fabrication
and DRAM technologies. He moved to Nagoya
Institute of Technology in 1996 and is currently
involved in education and research on semiconductor materials and device physics.

Vol. 11 No. 5 May 2013[25] http://www.ntt.co.jp/milab/doc/201305d019.pdf[26] http://www.ntt.co.jp/milab/doc/201305d020.pdf[27] http://www.ntt.co.jp/milab/doc/201305d021.pdf[28] Bellcore TR-332, “Reliability Prediction Procedure for Electronic
Equipment,” Issue 6, 1997, Telcordia SR-332, Issue 2, 2006.
[29] PRISM, User Manual ver 1.3, RIAC, 2001.
[30] IEC/TR 62380, “Reliability Data Handbook––A Universal Model for
Reliability Prediction of Electronics Components, PCBs and Equipment,” 2003.
[31] J. B. Bowles, “A Survey of Reliability-Prediction Procedures for
Microelectronic Devices,” IEEE Trans. on Reliability, Vol. 41, No. 1,
pp. 2–12, 1992.
[32] M. G. Pecht and F. R. Nash, “Predicting the Reliability of Electronic
Equipment,” Proc. of the IEEE, Vol. 82, No. 7, p. 992–1004, 1994.
[33] J. W. McPherson and C. F. Dunn, “A Model for Stress-induced Metal
Notching and Voiding in VLSI AI-Si Metallization,” J. Vac. Sci. &Technol. B.5, pp. 1321–1325, 1987.
[34] N. Shiono, T. Mizusawa, K. Hiraoka, and S. Kanamori, “Reliability of
0.5 μm BiCMOS Gate Array LSIs,” NTT R&D, Vol. 41, No. 9, p.
1095, 1992 (in Japanese).
[35] O. Fujita, Y. Nakano, and G. Iwane, “Reliability of semiconductor
lasers for undersea optical transmission systems,” IEEE Trans. Electron Devices, Vol. 3, No. 6, p. 1211–1216, 1985.
[36] A. S. Jordan, “A Comprehensive Review of the Lognormal Failure
Distribution with Application to LED Reliability,” Microelectron
Reliab., Vol. 18, No. 3, p. 267–279, 1978.
[37] H. Satoh, et al., “Semiconductor Devices for CS-36 M Deep Water
Submarine Cable Repeaters,” NTT R&D, Vol. 23, No. 3, p. 507, 1992
(in Japanese).
[38] ITRS (International Technology Roadmap for Semiconductors),
PIDS 2009.

Shin’ichiro Mutoh

Project Manager, First Promotion Project, NTT
Microsystem Integration Laboratories.
He received the B.E. and M.E. degrees in electronic engineering from Chiba University in1986 and 1988, respectively. In 1988, he joined
NTT, where he was initially engaged in researchon ultrahigh-speed BiCMOS SRAM and lowpower CMOS SRAM circuits. He was also
involved in R&D of low-power and low-voltage
digital circuit technologies for mobile phone/equipment applications, especially multiplethreshold-voltage CMOS (MTCMOS) circuits.
His current interests are ultralow-power wirelesssensor node architecture and circuit design foradvanced sensor network systems and LSI designfor optical telecommunication networks. He
served as a Program Committee Member of the
Symposium on VLSI Circuits from 2008 to 2011.
He is a member of the International Solid-State
Circuits Conference (ISSCC) Technology Direction Subcommittee, the IEEE Asian Solid-State
Circuits Conference (A-SSCC) Emerging Technology and Applications and the JSAP Executive
Committee of Symposia on VLSI Technology
and Circuits. He is also the Director, General
Affairs of IEICE Electronics Society.

12Global Standardization Activities

Overview of MPLS-TPStandardization
Makoto Murakami and Yoshinori Koike
Abstract

The recent prevalence of IP (Internet protocol) based services has increased the demand for transport
network technology that can efficiently accommodate both packet-based and circuit-based traffic whileachieving almost the same operation and maintenance capabilities as traditional transport networks suchas synchronous digital hierarchy or optical transport networks. Thus, ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) has developed a packet transport technology called MPLS-TP (multiprotocol label switching-transport profile) in collaboration with IETF (Internet Engineering Task Force).
This article introduces the overview and history of MPLS-TP standardization.

1. Outline of MPLS-TP technology
Telecommunication carriers have employed transport technologies standardized by ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) including synchronous
digital hierarchy (SDH) or optical transport networks(OTNs) in their core networks that convey large
capacity traffic over long distances. Recently, apacket-based technology that can accommodate
packet-based data more efficiently than the current
SDH or OTN has been demanded because packetbased traffic has become dominant in these networks
as the number of Internet protocol (IP)-based services
has increased. However, traditional packet network
technologies have been problematic for telecommunication carriers to adopt in networks because of alack of sufficient OAM (operation, administration,and maintenance) functions, fault localization, andfast switching in the event of a failure. Thus, a new
packet-based transport network technology that has
the same operation and maintenance capabilities as
traditional transport network technology is required.
One of the significant features of packet transport
network technology is that a path for packets between
two network elements (NEs) is explicitly determined,
and the connectivity of the path is periodically monitored to allow network operators to manage the path

condition; this is called being connection-oriented.
Other key functions should include protection switching, which promptly recovers a particular path when
the route of the path has a fault; alarm transfer, which
promptly transmits information about faults to otherNEs; and traffic engineering, which enables operators
to flexibly assign a bandwidth on a path.
MPLS (multiprotocol label switching) has beenstandardized in IETF to make it possible to explicitly
determine the path route of IP packets by attaching alabel to an IP packet and forwarding the packet byinspecting only the label instead of the IP address
header itself, as shown in Fig. 1. MPLS, however,does not have sufficient maintenance operation functions as required in transport networks, and thus,
some inherent features, including PHP (penultimate
hop popping), ECMP (equal cost multi-path), andlabel merging could disrupt the management of aconnection-oriented path between the two end points
because of a lack of traceability. Moreover, MPLS
technology makes it difficult to secure user data traffic in telecommunication carrier networks. This isbecause all the paths in MPLS are basically controlled by the control plane (part of the autonomous
control) through a soft state procedure in which any
fault that occurs during the exchange of control messages about a path causes a disconnection of the pathsin the data plane even though they are not experiencing
NTT Technical Review

Global Standardization Activities

Attach label to an IP
packet.

Connection-oriented path
IP/MPLS networkIP
IP router

IP 10

IP 20

IP
MPLS
router Label switching

Remove label fromIP packet.

transfer

Fig. 1. Data forwarding in MPLS label switching.

IP/MPLS

MPLS-TPExisting functions that do not satisfy
transport requirements:- PHP- ECMP- Label merging/LSP merge- LDP/non-TE LSPs
- IP address forwarding

Existing functions
available in transport
networks

Architecture- Isolation of data plane from control plane- MPLS/PWE3 architecture- MPLS forwarding- GMPLS/PWE3 control

OAM- In-band OAM channel- Performance monitor- Tandem connection- Alarm notification and transfer

Protection- High-speed switching(< 50 ms)
- Linear & ring topology

MPLS label allocation

Label Value |TC|S|TTLMPLS header

Additional functions

GAL(13) |TC|S|TTL 0001|ver|rsv|
MPLS header

GMPLS: generalized MPLS
LDP: label distribution protocol
LSP: label switched path
NMS: network management system
PWE3: pseudo-wire emulation edge-to-edge
rsv: reservation

Operation- NMS operation- Static provisioning- Traffic engineeringchannelACH-TLV
type

ACH
S: bottom of stack
TC: traffic class
TE: traffic-engineered
TTL: time to live
ver: version

Fig. 2. Differences between IP/MPLS and MPLS-TP.

a failure.
In contrast, MPLS-TP emphasizes operation andmaintenance capabilities by improving the data plane
control mechanism, introducing static and central
operations by the management plane, and implementing various OAM functions and high-speed protection switching, while some MPLS features that could
disrupt transport network operation are disabled, as
shown in Fig. 2. A GAL (generic associated channel
Vol. 11 No. 5 May 2013label) is newly defined in the MPLS header stack for
identifying the OAM packet, and each OAM function
is differentiated by an ACH (associated channel).
MPLS-TP also enables the paths in the data plane tobe controlled by the management plane and allows
network operators to manually and intentionallymanage all the paths. Moreover, when a control plane
is introduced, MPLS-TP employs an ASON (automatically switched optical network), described inGlobal Standardization Activities

From L3

ITU-T

IETF

IP
MPLS

Aims to develop apacket transporttechnology as highly
reliable as traditionaltransport networks on
the MPLS forwardingarchitecture.

MPLS-TP
conflict
MPLS-TP

Aims to develop a packet
transport technology byadding some functions
and making minimumchanges to current
MPLS implementations.

Ethernet
From L1OTN (WDM)

Fig. 3. Differences between ITU-T and IETF concepts.

ITU-T Recommendation G.8080, which isolates
paths in the data plane from any failure in the control
plane so that a fault in the control plane never affects
the user data traffic. Thus, MPLS-TP can provide
highly reliable network services, which is one of themost significant attributes required for transport networks.
As such, MPLS-TP was created as a merger of thetransport network technology mainly standardized by
ITU-T and MPLS technology mainly standardized by
IETF. This merger has brought out some differencesin the concepts of the two standardization organizations, as shown in Fig. 3. ITU-T’s aim was to develop
packet transport technology with the high reliabilityof traditional SDH or OTN transport networks bycreating a new mechanism on the MPLS forwardingarchitecture irrespective of existing MPLS implementations. By contrast, IETF has persisted with thecurrent MPLS implementations in their aim to develop an MPLS-TP technology with minimum changes.
Therefore, the MPLS-TP specifications tended todeviate from the original concept and to be complicated due to the constraints from the existing MPLS
implementations. Another point in the controversy isthat IETF follows a process called rough consensus,
in which there are no definite rules in the decisionmaking process in developing their standard RFCs
(requests for comments), while ITU-T has clearly
defined rules for approving Recommendations basedon the unanimous agreement among member states
as an affiliate in the United Nations. With these backgrounds, MPLS-TP standardization has created amajor controversy between the organizations.



2. History of MPLS-TP standardization
2.1 2005 to 2010
ITU-T began standardizing T-MPLS (TransportMPLS) in 2005. This standardization effort introduced OAM and protection functions to the MPLS
data forwarding mechanism in order to satisfy packet
transport network requirements and was based on
ITU-T’s experience in developing standards for transport network technologies. ITU-T approved the main
T-MPLS recommendations in 2006 and 2007; theseinclude G.8110.1 (Architecture of Transport MPLS
(T-MPLS) layer network), G.8121 (Characteristics ofTransport MPLS equipment functional blocks),
G.8112 (Interfaces for the Transport MPLS (TMPLS) hierarchy), and G.8131 (Linear protection
switching for Transport MPLS (T-MPLS) networks.
However, IETF claimed that T-MPLS did not conform to the existing MPLS specifications when ITU-T
had made progress on the LC (last call) of G.8114(Operation & maintenance mechanism for T-MPLS
layer networks) and G.8113 in January 2008. In
anticipation of future collaboration with IETF on this
topic, ITU-T stopped the LC of G.8114 and established a Joint Working Team (JWT) with IETF sothey could cooperate on the packet transport network
standardization in February 2008.
In December 2008, after T-MPLS was renamed as
MPLS-TP, ITU-T and IETF agreed to publish some
IETF RFCs based on T-MPLS Recommendations byJune 2009, and they therefore developed several
MPLS-TP RFCs, including those regarding theframework and requirements.
At the 74th IETF meeting in March 2009 ITU-T
NTT Technical Review

Global Standardization Activities

experts asserted their intention to employ the OAM
tools defined in Y.1731 for MPLS-TP because they
had already been proven to meet the requirements ofthe transport network and were sufficiently maturefor deployment. At the 75th IETF meeting in July
2009, however, the IETF MPLS working group tooka unilateral decision to give priority to backwards
compatibility with existing IP/MPLS protocols andsaid that they would develop and support only one
solution based on extensions of bidirectional forwarding detection (BFD) and label switched path
(LSP)-ping, which they had been developing for IP/
MPLS.
The ITU-T experts regarded the complex OAM tool
sets developed for compatibility with the existing
IP/MPLS implementations preferred by IETF as adeviation from the primary MPLS-TP concept.
Therefore, at the SG (Study Group) 15 plenary inJune 2010, ITU-T decided, with support from manymember states, to develop an MPLS-TP OAM Recommendation, G.8113.1 and to proceed with thestandardization of both types of OAM tools to satisfy
the demands of ITU-T and IETF. ITU-T also requested IETF/IANA (Internet Assigned Numbers Authority) to assign a codepoint to differentiate these two
OAM solutions. In August 2010, the ITU-T TSB(Telecommunication Standardization Bureau) director had a direct talk with the IETF chairman to settle
the controversy, but they were unable to reach a resolution. The IETF chairman thus unilaterally declaredthat IETF would not accept any other OAM solutions
than those developed by IETF.
2.2 2011
At the SG15 plenary in February 2011, the approval of two OAM Recommendations, G.8113.1 advocated by ITU-T and G.8113.2 by IETF, were discussed, but the progress of the meeting stalled due toan intense dispute. Finally, the ITU-T SG15 chairman
decided to move G.8113.1 forward to the approval
process based on the result of a vote among member
states, although it was exceptional to take such action
at an ITU-T SG meeting. Then, IETF issued a comment in a newsletter criticizing the decision made in
ITU-T [1], and ITU-T also issued an article titled“THE FACTS” in the ITU-T newslog to reveal thehistory of MPLS-TP standardization [2]. Hence, theserious conflict between the two organizations cameto light. Japanese telecom carriers and vendors had
dedicated discussions in the TTC (Telecommunication Technology Committee) in Japan to decide theposition of Japanese member states on the MPLS-TP
Vol. 11 No. 5 May 2013

standardization, because any final decisions must be
made at the member state level in ITU-T.
In June 2011, a circular letter was issued by the
ITU-T TSB asking member states to respond to an
inquiry on proceeding with the traditional approval
process for G.8113.1. With support from 33 member
states including Japan and against opposition from 5
member states, it was decided that G.8113.1 would
move forward for approval at the following SG15
plenary. Since a further dispute at the SG15 plenary
was expected, Japan submitted a contribution to propose a compromise as a member state in a relatively
neutral position after holding repeated discussions
with the stakeholders including the ITU-T bureau, theSG15 chairman, and management members. However, G.8113.1 was finally disapproved at the SG15
plenary held in December 2011 due to opposition
from 4 member states (the USA, Israel, the UK, andFinland) that was unable to be resolved despite theintense formal and informal discussions and negotiations conducted throughout the day and night. As aresult, the SG15 Chairman declared a deadlock in theapproval of Recommendation G.8113.1, and the decision was committed to the WTSA (World Telecommunication Standardization Assembly)-12 held in
November 2012. The ITU-T TSB director issued acomment on the ITU-T newslog in which he gave adetailed account of the event and expressed his great
appreciation to the member state of Japan for its
efforts to settle the situation [3].
2.3 2012
NTT participated in further discussions on MPLSTP standardization in CJK (China Japan Korea)
meetings held with Chinese and Korean experts whoshared the same opinion and submitted a common
proposal document by the three countries to move
G.8113.1 forward as well as a series of related MPLSTP Recommendations at the APT (Asia-Pacific Telecommunity) meeting held in August 2012. The proposal was supported at the meeting and approved to
be an APT common proposal to WTSA-12.
At the SG15 meeting in September 2012, a degreeof progress was made, as some MPLS-TP Recommendations were approved or consented to. A proposal from Canada that G.8113.2 should also beapproved in WTSA-12 was agreed, and the approvalof both draft Recommendations became a matter for
simultaneous discussion at WTSA-12.
At the beginning of the WTSA-12 meeting held in
November 2012, these two draft Recommendations
were simultaneously submitted for approval and wereGlobal Standardization Activities

ITU-T basis

IETF basis

Architecture

G.8110.1

InterfaceG.8112Equipment function block (generic)

G.8121

Not yet approved

Equipment function block (specific)

G.8121.1

G.8121.2

OAM

G.8113.1

G.8113.2

Linear protection

G.8131.1

G.8131.2

Ring protection

G.8132Equipment management (generic)

G.8151Equipment management (specific)

G.8151.1

Approved or consented(in AAP process)G.8151.2

Management Information

G.8152

Terminology

G.8101DCN

G.7712

AAP: alternative approval process
DCN: data communication network

Fig. 4. Structure of MPLS-TP Recommendations and current status.

successfully approved with no opposition. Finally,
IANA formally notified ITU-T that it would assign an
ACH codepoint for identification of an OAM packet
based on G.8113.1, and the two OAM solutions
respectively preferred by ITU-T and IETF finallycame to be international standards.

including equipment functional blocks and protection
schemes will be further discussed and approved.
Additionally, some other significant issues related to
MPLS-TP including a point-to-multipoint connection scheme and layer integration are expected to be
standardized in the future.

3. Future activities

References

The controversy of MPLS-TP standardization that
continued for more than seven years since the standardization of T-MPLS was successfully resolved inpart because of the work done to gain approval of the
OAM Recommendations at WTSA-12. The status ofinternational standardization of MPLS-TP is now asshown in Fig. 4. Other draft Recommendations



[1]

[2]
[3]

Internet Society Newsletter.
http://www.internetsociety.org/articles/ietf-and-internet-societystatement-relating-today%E2%80%99s-itu-t-sg15-decision-willlead-non
ITU-T Newslog.
http://www.itu.int/ITU-T/newslog/MPLSTP+The+Facts.aspx
ITU-T Newslog.
http://www.itu.int/ITU-T/newslog/Carrier+Network+Standards+App
roved+At+Geneva+Meeting.aspx

NTT Technical Review

Global Standardization Activities

Makoto Murakami

Senior Research Engineer, NTT Network Service Systems Laboratories.
He received the Ph.D. degree in electrical engineering from the University of Tokyo in 2009. He
joined NTT in 1988 and initially engaged in
R&D of long haul transmission systems using
optical amplifiers and coherent modulation/
demodulation schemes at the emergence of thosetechnologies. After completing development anddeployment of a commercial optically amplified
submarine system, he continued R&D of WDM(wavelength division multiplexing) systems tofurther increase the fiber transmission capacity.
From 2001 to 2003, he worked for NTT Communications, where he was engaged in construction and operation of international communication networks mainly in the Asia-Pacific area.
Since 2003 he has been an active participant in
ITU-T SG15 and has been involved in R&D oflarge-capacity photonic network systems andstandardization of optical transport networks. He
is currently the chairman of the transport networks and EMC WG in TTC, Japan.

Vol. 11 No. 5 May 2013

Yoshinori Koike

Senior Research Engineer, First Promotion
Project, NTT Network Service Systems Laboratories.
He received the B.S. and M.S. degrees from the
University of Tokyo School of Engineering
Department of Applied Physics in 1998 and
2000, respectively. Since joining NTT Network
Service Systems Laboratories in 2000, he has
been engaged in R&D of optical transmission
systems such as SDH, WDM, and OXC. Since
2003, he has actively participated in ITU-T SG15
activities concerning SDH, OTN, ASON, andsynchronization and packet transport technologies. He is currently an associate Rapporteur for
Question 3/15 (General characteristics of opticaltransport networks) of ITU-T SG15. He is amember of the Institute of Electronics, Information and Communication Engineers.

Information

NTT Establishes NTT Innovation Institute, Inc.
as its New R&D Center in North America

On April 1, 2013, NTT Innovation Institute, Inc. (NTT I3) was established in North America as a new research and development (R&D) centerin line with NTT’s “Toward the Next Stage” business strategy, which wasannounced on November 8, 2012. NTT I3 was established as a venue to
speedily develop world-class security cloud technology and from which tosupply the related services to the North American market, where the competition is the most intense.
At NTT I3, the aim is to develop services by combining the technicalcapabilities of the Institute with know-how (IP: intellectual property) andto promote open innovation in cloud services, which are changing very
rapidly and require swift deployment in the market.
Specifically, the service development in North America will be promoted
through security and cloud research and development. Additionally, thesuccessful deployment of “global cloud services”, a growth driver of the
NTT Group, will be strengthened.
In addition, the cloud services refined in North America can be deployedin Japan and in newly emerging markets as modularized IP.
At a kick-off meeting held at NTT I3 on April 5, Hiromichi Shinohara,
NTT Executive Vice President and Director of the Research and Development Planning Department, offered greetings saying, “We expect NTT I3
to play a major role in accelerating NTT’s global business.”
The newly appointed Chief Operating Officer (COO) of NTT I3, Eiji
Kuwana (formerly, General Manager of NTT Secure Platform Laboratories), went on to say, “We must aggressively take up the challenge of making the most of new business opportunities.”
These words underscore the resolve of the entire NTT I3 staff to becomea core base in strengthening the global R&D of the NTT Group.

Greetings from COO Eiji KuwanaNTT Technical Review