NTT Technical Review
November 2013 Vol. 11 No. 11
View from the Top
Mitsuyoshi Kobayashi,
Senior Vice President,
Director of Technology Planning Department,
Director of Strategic Business Development Division, NTT

Front-line Researchers
Naonori Ueda
Director of Machine Learning Data Science Center,
Senior Distinguished Scientist,
NTT Communication Science Laboratories

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data
Applications of Big Data Analytics Technologies for
Traffic and Network Management Data––
Gaining Useful Insights from Big Data of Trafficand Network Management
Analyzing Internet Traffic Structure
through Big Data Technology
Traffic Management of Video Streaming Services
Network Failure Detection and Diagnosis byAnalyzing Syslog and SNS Data:
Applying Big Data Analysis to Network Operations
Analyzing Macroscopic Demand Structure Based on
New Model of S-shaped Growth

Regular Articles
Fiber-mounted Electro-optic Probe for Microwave
Electric-field Measurement in Plasma Environment

Global Standardization Activities
International Standardization Trends in China

External Awards
External Awards

View from the Top

Thinking and Trying while Running;
Heading toward the Inter-service Era
as a Value Partner
Mitsuyoshi Kobayashi,
Senior Vice President,
Director of Technology Planning
Department,
Director of Strategic Business
Development Division, NTT
Overview

Mitsuyoshi Kobayashi, Senior Vice President and Director of the Technology Planning Department ofNTT, says that we need to adopt a new corporate mindset that places importance on the new inter-service
era and the challenges that this market brings. We asked him about NTT’s Medium-Term Management
Strategy and how he is preparing to work in a telecommunications industry that demands speed and newvalue creation.
Keywords: telecom, strategy, resilient network

The smartphone and cloud as the lead players of thisera—new business opportunities in the market—Mr. Kobayashi, how would you describe the current
conditions in the telecommunications industry?
The speed at which technology is advancing is simply amazing, and foremost among this phenomenon
is the rapid expansion of the smartphone market. Far
from being a gradual development, the increase insmartphone sales seems to have occurred all at once.
The smartphone, so to speak, is a mobile computer.
Up to recently, the mobile phone had just been a tool
for making calls, but it is now evolving into a solutions terminal that can resolve problems in accordance with customer desires and needs.


Last fiscal year, sales of NTT DOCOMO terminals
reached approximately 24 million units, with morethan half of that being smartphones. This trend is only
getting stronger, and sales are expected to increase bynearly 70% this fiscal year.
The lead player in the telecommunications industry
is changing completely from fixed to mobile communications. According to an information-communications white paper published this year, the numberof fixed broadband subscribers in Japan, which canbe counted among the world’s top broadband countries, is somewhat static at approximately 35 million,
including optical and other types of broadband services. However, mobile broadband in Japan isincreasing at an annual rate of more than 10 million
contacts centered around mobile phones.
NTT Technical Review

View from the Top

The number of mobile broadband subscribers at theend of the last fiscal year was around 140 million,
which is more than the population of Japan*1 andthereby greater than the rate of one phone per person.
It can therefore be considered that mobile broadband
subscribers are entering a saturated state, but new
ways of using mobile broadband are neverthelessbeing thought up. For example, if automobiles areequipped with communication functions, we canenvision that the number of mobile broadband subscribers will increase by several tens of millions, anda wide variety of related services will be created.
Consequently, we can expect mobile services toexpand even further in the years to come.
We must also note the growth of the cloud. It’s not
simply that corporate customers are coming to use thecloud in increasing numbers, but also that many of theservices that you and I use on a daily basis are utilizing the cloud.
Up to now, it appears that carriers like us havemainly been concerned with thinking about and providing services, but over-the-top (OTT) operators
have recently entered the market to provide services
that combine cloud technology and smartphones. For
example, applications that provide free voice calls
such as LINE are now appearing, with the result thatvoice calls are simply becoming an “added bonus.”
This is a devastating development for telecommunication carriers like NTT, for which revenues fromphone services have been a major foundation of theirbusiness.
Another development is that traffic is increasing
annually owing to the advances made in fixed andmobile broadband services. Traffic from mobile communications in particular is doubling every year. We
can attribute this increase in traffic to growth invideo-related services. The NTT Group provides
video services such as FLET’S TV, Hikari TV, anddVideo, and the total number of subscribers for video
services as of the end of July 2013 was approximately 10 million, which is twice that of 2012.
This means that network capacity must expand bythat amount, but under the present system of flat-rate
subscribers, NTT and other telecommunication carriers cannot expect to increase their earnings in proportion to the amount of communications traffic. In otherwords, we are faced with a situation in which anincrease in traffic is not reflected in revenues.

*1	127.3 million as of October 2013 by Statistics Bureau, Ministryof Internal Affaires and Communications

Vol. 11 No. 11 Nov. 2013

Medium-Term Management Strategy“Towards the Next Stage”—It’s natural for a business enterprise to think about
revenue. How do you plan to deal with this issue?
Our business model and service format must change
rapidly to keep up with the times. The NTT Group
adopted a new system last fall and formulated a
Medium-Term Management Strategy called “Towards
the Next Stage.”
Although it’s still in the planning stage, a key element of this strategy is the concept of “inter-service,”
which I will believe will become our guidepost going
forward. It’s been nearly 20 years since the birth ofthe modern Internet (considered to be 1995) and thefirst time that the Internet Explorer browser was
installed as standard in personal computers (PCs).
Since then, the Internet has spread explosively andbrought about major changes in our daily lives andcorporate business models. When I think about what
the Internet could become in the future, I can see that
there is still lots of room for improvement.
One area for improvement is ease of use. At present, various types of operating systems (OSs) exist
for mobile terminals and PCs, which forces service
providers to prepare services that match each OS anddevice type. To get many customers to use a service,
it must be developed and maintained for each OS. In
addition, a number of telecommunication carriers
exist, and NTT customers, for example, can only use
services connected to the NTT network. The types ofterminals on which a service can be used are also
limited. For customers who would like to enjoy avariety of services, this situation can hardly be called
convenient.
In the beginning, the meaning of the Internet was toView from the Top“connect” or “join together” disparate networks.
Now, from the customer’s point of view, the prefix“inter” will come to mean more than just the physicalconnection of networks—it will also mean the open
and seamless connection of services on those networks and the OS-free and device-free use of thoseservices. This is the inter-service concept.
However, if the NTT Group were to take up thiskind of business, the result would be similar to a
Ptolemaic (geocentric) system. Beyond our corporate
customers are general customers, and if we were toattempt to provide new services in this field, it would
require an around-the-clock effort. Going forward,
we should aim to expand in business areas in which
we assist our corporate customers as a value partnerin creating new services, and at the same time, tosearch out the best match between different services.
It often happens that companies and general customers are ignorant of just what types of services exist, so
we can play a role here as an intermediary providing
a “connection” function, that is, in finding an optimal
service or even multiple services that satisfy a customer’s needs and provide greater value all around.
This is the type of inter-service world that I would
like to create.
Making service-to-service and customer-tocustomer connections
In terms of concrete actions, we are beginning by
exploring various ideas, for example, “online tooffline.” In this effort, we aim to provide new added
value by linking the network world with the realworld.
To take the case of shopping as an example, theconventional way is to go to a retail shop in person
and purchase desired products. These shopping
actions are usually done independently from those
performed when purchasing products on the Internet.
However, by combining these two forms of shopping,
we can create new added value. Specifically, we canprovide the consumer who is searching for productson the Internet with information on where to go toactually see those products in person, and at the sametime, we can provide retailers with information on thefact that such customers exist and on what products
they find interesting.
One example we are now working on is a link-up
between Tokyo Metro and Seven & I Holdings Co.
through an information delivery service using a Wi-Fi
platform. Here, we install a Wi-Fi antenna in each 7Eleven convenience store and provide services uniqueto each store. For example, we can enable musicalcontent of a popular singer to be downloaded only ata specific store, and we can provide information onthat service at a nearby train station. This should havethe effect of encouraging customers to visit that store,
which should, in turn, promote consumer activity,that is, product purchases, while they are downloading content using the Wi-Fi service.
Since Wi-Fi can be used irrespective of the type ofterminal or OS, this service can be provided even to
customers who receive their telecommunication service through other carriers. Using such an open anddirect, carrier-independent means of access can open
up lots of business opportunities. In this way, we are
exploring ways of making service-to-service and customer-to-customer connections to uncover moreinter-service possibilities.
Building a resilient, flexible, and strong network—How do you feel about the idea that a businessstrategy should consist not only of entering new fields
but also of fortifying existing capabilities?
I would like to think first about a service-driven
network. Here, I don’t actually mean thinking about
services over the network but rather about creating aneasy-to-use network that service-providing companies will want to use.
As for fixed communications services, we cannot
expect a huge increase in the number of subscribers.



NTT Technical Review

View from the Top

Nevertheless, video services such as those that provide high-quality TV programs must be able todeliver large amounts of information, and developing
optimal optical broadband technology for thispurpose is still a very active field.
We must also keep in mind that the mobile network
employs a fixed optical fiber network running frombase stations to the core network. I believe that this
optical fiber network will become increasingly
important from here on.
Of course, we can hardly think that customers utilize only the fixed network, so we must think about
ways of forming tie-ups with the mobile network.
There are regulations covering the provision by
NTT of combined fixed and mobile services, so doing
so is unfortunately difficult at present. From the customer’s point of view, however, I think that a world inwhich services could be enjoyed over an optimal network regardless of whether they are fixed or mobile
would certainly be desirable.
In the wake of the Great East Japan Earthquake, wemust think about a stable, safe, and secure networkthat is robust to disasters and resilient to sudden
jumps in traffic as well as an energy-saving networkthat is less susceptible to nuclear power plant problems and steep rises in fuel costs.
We must also consider the need for a network thatcan provide low-rate services. An expensive network
will not be used, and value comes out of using a network. We need to continue our efforts in making the
NTT network all the more competitive.
The NTT business companies are already taking
various measures in this regard, but to enhance ourability to provide a highly competitive network, wemust look to NTT research and development (R&D).
One way of radically lowering costs is to incorporate
virtualization and SDN (software-defined networking) technologies, which are now being studied. I
think these technologies will be the focus of evenmore research activity in the future.
Furthermore, considering that fixed, mobile, anddomestic/overseas networks have so far been constructed independently by the various NTT businesscompanies, we again look to R&D to drive the construction of an optimal network that applies technologies to be used by the entire group and that minimizes
costs. I am confident that all concerned will show ahigh level of leadership in this regard.

Learning from the words of Hakuin Zenji:
Meditation in the midst of activity is a thousand
times superior to meditation in stillness.
—What do you think is an essential element of a business strategy in today’s world?
In today’s rapidly changing world, it goes without
saying that responding to the market with a sense ofurgency is an absolute necessity. Those who stop andhesitate and fail to think and come up with answers
while they are running will be left behind in no timeat all in this era.
A dynamic and quick response is certainly the key.
In my case, moving suits my temperament more than
thinking. In relation to this, the NTT Group once
attempted to develop PC equipment at a time whencustom-made PCs were appearing. “Custom-made”
was an idea that was not born out of conventional
NTT thinking.
By grasping the times and making an actual move,
we were able to obtain valuable feedback on our customers’ way of thinking and advanced business models. Unfortunately, the result was not that great, but
the know-how cultivated from this endeavor is still
nourishing the NTT Group and myself as well. Try
thinking of a moving shinkansen train carrying customers in the current era. When viewed from a fixed
point, it is very hard to make out who or what is riding
on the train. To do so, you have to run parallel to thetrain at the same speed or even faster.
I once learned from a certain individual the wordsof Hakuin Zenji*2, a famous Japanese Zen master:*2 Hakuin Ekaku (1686–1768) was one of the most influential figures in Japanese Zen Buddhism.

Vol. 11 No. 11 Nov. 2013

View from the Top“Meditation in the midst of activity is a thousand
times superior to meditation in stillness.” Certainly,
this means that testing out acquired wisdom whilemoving enables one to acquire new wisdom and makeit a part of oneself. These words express the stancethat we need to take in this era.
—Mr. Kobayashi, can you leave us with a few words
for everyone in the NTT Group?
At the risk of putting on airs, I live by thosewords—I feel that it is important to think while running. Even if a variety of constraints are said to exist,
going ahead and trying something instead of worrying about those constraints and doing nothing will at
least enable you to learn something about those
restrictions. In fact, you may find out that those constraints were not such a problem after all. Taking up
the challenge of achieving what has previously beeninconceivable may actually bring change to theworld. This doesn’t mean we should have a reckless
attitude, but it does mean that we should run in a
focused manner in a common direction.
Interviewee profile
Career highlights
Mitsuyoshi Kobayashi joined Nippon Telegraphand Telephone Public Corporation in April 1982.
He served as Senior Manager of the Personnel
Department of NTT WEST beginning in 2002, as
General Manager of the Okayama Branch of NTT
WEST beginning in 2006, and as a Senior Vice
President and General Manager of the Service
Management Department of NTT WEST beginning in 2008. He took up his present positions inJune 2012.

NTT Technical Review

Front-line Researchers

Applying Big Data in Innovative Waysas the Leader of a New
Interdisciplinary Research Center
Naonori Ueda
Director of Machine Learning
Data Science Center,
Senior Distinguished Scientist,
NTT Communication Science Laboratories
Big data has been attracting a great deal of attention in recent
years, but there is still no real sense of the role it can play in society. NTT is now taking a giant step toward the widespread, practical application of big data analysis. Senior Distinguished Scientist
Dr. Naonori Ueda is an active pioneer in the research of machinelearning. We asked him about his career in research and about theMachine Learning Data Science Center that he directs.
Keywords: big data, machine learning, statistics

From an early interest in statistical learning theory
to becoming an expert in machine learning—Dr. Ueda, please tell us about your research career
to date.
After receiving my M.S. degree in communicationengineering from Osaka University in 1984, I entered
the Electrical Communication Laboratories of Nippon Telegraph and Telephone Public Corporation in
Yokosuka, Japan. At that time, I was assigned to alaboratory involved in the research and development(R&D) of image-processing and pattern-recognition
technologies, but since the group was right in the
middle of using their research results in practicalapplications, and as I was just a newcomer, I was
relegated to doing tasks outside research such as taking the minutes of meetings and carrying magnetic

tapes to other research sites. During whatever freetime I had, I read books and papers on specialized
topics to learn as much as I could, and I found myself
yearning to become more involved in pure research.
One year later, Nippon Telegraph and Telephone
Public Corporation was taken private to becomeNTT. I was given the task of covering up labels saying “Nippon Telegraph and Telephone Public Corporation” on a huge number of devices and fixtures withstickers having the name “NTT.” I remember withsome fondness how I was buried under with this workfor days on end and how I even surprised, if notangered, some female employees when I crawled
under their desks to affix these stickers. NTT R&D
was also targeted for restructuring, and with theestablishment of NTT Human Interface Laboratories
in 1987, some personnel including myself were transferred there, and I was assigned to do basic research
NTT Technical Review

Front-line Researchers

Technology for giving learning capabilities to machines(information processing system)
Input
Characterrecognition

Image

Information processing
system

7210
Characters
こんにちは

Speechrecognition
Machinetranslation

OutputAudio
signalText (sentence)

This is a pen.
English

これはペンです．
Japanese

General-purpose data analysis technology based on mathematical statistics and optimization theory(“General purpose” refers to the capability of targeting field-independent, unstructured data.)

Fig. 1. Outline of machine learning technology.

in the field of computer vision. Specifically, I took up
the theme of image understanding from line drawings, and it was at this time that I began to develop aninterest in statistical learning theory.
Then, in 1991, I moved over to the newly established NTT Communication Science Laboratories in
Keihanna and began doing basic research in the fieldof statistical machine learning (Fig. 1). The next ten
years proved to be a very fulfilling time for me as I
was able to immerse myself in research with stays at
Purdue University in the United States as a visiting
scholar, the University College London in the United
Kingdom for short-term research, and elsewhere.
Then, beginning in 2010, I served for about three
years as director of NTT Communication Science
Laboratories. Finally, in April 2013, I became a
Senior Distinguished Scientist and in July, Directorof the newly established Machine Learning Data Science Center (MLC). I should mention here that MLC
was initially founded as a virtual organization in
April 2013 but was launched as a formal organizationin July 2013. My superiors said to me, “There are
great expectations that MLC will bring machinelearning and big data together and make a huge contribution to the R&D of big data applications at NTT.”
As director, I take my position very seriously andrealize that this is a heavy responsibility.
Machine learning as an optimal tool for creating
new value from the analysis of human behavior—What exactly is machine learning?

Vol. 11 No. 11 Nov. 2013

Machine learning was developed in around 1970 as
a research field within the area of artificial intelligence. Neural networks, by the way, which boomedin the 1980s, are a type of machine-learning technology that has come to be used in a variety of application fields. This boom died down some years later, butmachine learning, perhaps as a result of the fact thatthe boom died down, went on to evolve into a somewhat simple form of mathematics-oriented researchcalled learning theory.
Learning methods can be broadly divided into
supervised learning and unsupervised learning. An
example of supervised learning would be a mother(as teacher) instructing her child on how to discriminate between a hat and a ball by saying, “this is a hat
but that is a ball.” This type of learning has come to
be used in pattern recognition technology as in therecognition of hand-written characters. An exampleof unsupervised learning would be a child observing
what is common and what is different between a hat
and a ball on his or her own. Clustering, or groupingof similar data, which is a basic process in data analysis, can be thought of as unsupervised learning. Various types of cluster analysis have recently come into
use for such applications as determining what sorts ofcommunities have formed on a social networking siteor what sorts of shoppers are using an e-commercesite.
What is the difference between machine learningand conventional data analysis? Predicting the weather, for example, is a science that derives answers
based on the way in which the atmosphere changes,
for example, from the formation of certain types ofFront-line Researchers

General-purpose data analysis technology
based on mathematical statistics and optimization theory(“General purpose” refers to a wide range of base technologies including
pattern recognition, language processing, robot engineering,
BMI (brain-machine interface), bioinformatics, and data mining.)

In the big data era, targets and problems that cannot be analyzed
using only a strict physical model are common.

From a physical model to a goal-oriented model
Machine learning is promising as a base technology forservice science (methodology for enhancing service)

Fig. 2. Usefulness of machine-learning technology in the big data era.

Probabilistic modelp ( xn , zn ; Θ )D = { x1 , x2 , ... , xN }Observation data

Parameters
Observation
variables Latent variables

Generative model:
Hypothesis on how observation data were generated
in terms of a statistical model (series)
Latent variables: Unobserved hidden variables
Parameters: Variables that characterize the probability distribution
Learning: Inverse problem of inferring a model from observation data

Fig, 3. Statistical machine learning based on a statistical model.

clouds. Machine learning, however, is different inthat it obtains knowledge from many observations
much like a local fisherman would say, “If these kindsof clouds appear, it will rain tomorrow.”
Although some basic principles may be used,machine learning predicts the future by establishing amodel for the mechanism that generates the data
behind the observation data. This model is not necessarily true, which lies in contrast with science, inwhich the theories that it derives must hold true.
For example, if the process of modeling humanpurchasing behavior came down to the modeling of
humans themselves, it could very well take more than
100 years to come up with a good model. A much
faster approach would be to use observation data on
what kinds of people purchase what types of products

under what conditions to realistically determine what
actions an individual can be expected to take. This iscalled a data-driven approach.
Modeling in machine learning can consequently becalled a service science (science whose purpose is toenhance service), and in this sense, it has come to be
applied to big data applications that deal with a hugeand diverse amount of real-world data. In short,machine-learning technology has come to be viewed
as a useful tool in big data analysis (Fig. 2). There arevarious types of research approaches in machinelearning, but I myself am researching a statisticalmachine-learning approach based on statistical models (Fig. 3). With a statistical model, it becomes possible to make use of even vague information and to
make predictions in a statistical manner. I believe thatNTT Technical Review

Front-line Researchers

Research institutions outside
NTT Group companies

Application fields(security, traffic, CRM, etc.)

Key application fields for NTT business:
Expanded research in specialized fields

Database technology:
Diverse data-storage, compiling,and search techniques

Batch-data processing
platform technology

Online-data processing
platform technology

Machine-learningtechnology

Parallel distributed computing technology:
High-speed computing techniques forlarge-scale data
Machine learning technology:
Analysis technology for creating newvalue from latent information

CRM: customer relationship management

Fig. 4. Machine Learning & Data Science Center (MLC).

this is a useful technology for carrying out decisionmaking using big data.
Research in the field of machine learning predates
the big data era, and machine-learning technology
has, in fact, come to be used in a variety of fields suchas machine translation. Machine-learning technology
has also come to be used as an elemental technology
in voice-recognition systems such as NTT DOCOMO’s Shabette Concier (Talking Concierge) voiceagent and in portal-site recommendation systems. It
would be no exaggeration to say that machine-learning technology is a general-purpose technology for
information processing systems.
Aiming toward a new world merging Jubatus
and machine learning as Director of the new
Machine Learning Data Science Centerlaunched this spring—Please tell us about the Machine Learning Data
Science Center.
The term “big data” has come to be somewhat of abuzzword today, but MLC was established as an
R&D base within the NTT laboratories to create
innovative services from big data. It is an interdisciplinary organization bringing together researchers
from related fields that have up to now been dispersed
across a number of research laboratories. There areVol. 11 No. 11 Nov. 2013

various dimensions to big data such as data collection
and management and data security, but R&D at MLC
focuses on the analysis of big data. It is for this reason
that machine learning by itself is insufficient. To process a massive and diverse volume of data in an efficient manner, we are researching and developing, forexample, the combination of machine-learningtechnology with the Jubatus parallel distributed computing environment developed jointly by NTT laboratories and Preferred Infrastructure (PFI), a venturecompany (Fig. 4).
Our mission is twofold. First, we will brainstorm
with NTT Group companies as a technology consultant to determine how to analyze big data held insideand outside the NTT Group to create new value, andwe will develop new technologies for doing so. Second, we will create innovative general-purpose dataanalysis technologies in the context of big data. These
two parts of our mission have a mutually dependent
relationship. In the second part, the presentation of an
academic paper on a new data-analysis technology
does not mark the end of our work. The technology ismeaningless if it is not applied to a real-world problem and evaluated accordingly. In the big data era,
field trials take on even more importance.
We currently have about 20 researchers working at
MLC, which is hardly a great number, but I think wewill become a substantially larger organization as we
broaden our interaction with NTT Group companies
Front-line Researchers

going forward.
—The Machine Learning Data Science Center seems
to have received much attention at Open House
2013.
Yes, I received questions from reporters on the purpose of launching MLC at the NTT Communication
Science Laboratories Open House 2013 held in June.
There was also much interest in what types of analysis would be targeted by machine learning. To give a
specific example, we are collaborating with the
medical-care producer of the NTT Research andDevelopment Planning Department to develop a system for analyzing the health-checkup data of NTT
employees. We feel that such analysis will prove useful in health promotion and disease prevention for all
NTT employees.
I also received questions from reporters on the key
feature of MLC. As I mentioned earlier, the MLC is
made up of researchers from diverse technology
fields such as machine learning, parallel distributed
computing, and database technologies as well as
applied technologies like customer relationship management (CRM) and security, all specialties of NTT
laboratories. Big data analysis through inter-field collaboration is the strength of MLC.
Creating scenarios with a focus on social
problems so that big data does not die out asa temporary boom
—What is the future outlook for big data analysis?
One difficulty in big data analysis is that the outcome of such analysis is unknown. For example,
when research is done on a new device, the specifications for that device may be set beforehand as target
values, and work to meet those values can then be
carried out, but in big data analysis, my colleagues
and I must from the start create a basic analysis scenario that describes what it is that we want to do. The
big question here is whether such a scenario can help
deal with social issues such as disaster prevention.
Simply refining an elemental technology will notbring forth a scenario with a sense of innovation. I
would like to hold discussions with group companies
that have abundant channels to customers and society
with the aim of contributing to the solutions of serious problems and creating value through big data
analysis.
Up to now, researchers in the field of machinelearning have led a somewhat behind-the-scenes existence, and they have had a comparatively high degreeof freedom in their work. Now, in the big data era,
they suddenly find themselves in the spotlight andthrust onto the front stage. This is essentially a boost
for researchers, but it also means that they now haveto deal with questions like “What can you do withyour research?” and “What have you accomplished?”
If researchers produce no truly useful results, big data
will come to an end as just a temporary boom. I
believe it is necessary for us to use the current flow in
big data as a tailwind and work aggressively to refine
our technology with the aim of making importantsocial contributions.
Up to now, I myself have been engaged in basic
research in the field of machine learning without any
specific plans for taking on big data. Perhaps I am
therefore an amateur of sorts, but in this era of big
data, my aim is to determine how technology can beused to advance society as a major issue in MLC.
Playing golf—a time to forget about researchmatters and enjoy one’s favorite hobby—Can you tell us how you spend your free time? Do
you have any hobbies?
For researchers, the work of research makes it difficult to differentiate clearly between weekdays andholidays and to keep research-related matters out of
one’s mind. I have said that scenarios are essential tocreating new value from big data, but that requires
imagination. Since technical books are hardly asource of imagination, I try to read a wide variety ofbooks to expand my interests and broaden my values.
I’m not sure whether this can be called a “hobby” assuch.
My true hobby is golf. Although I only get out to agolf course about once a month, I also enjoy researching golf theory by reading good books about it andfinding information on the Internet. I also played tennis during my student days, but I developed an interest in golf under the influence of my father and older
brother. Playing golf was always enjoyable if only forthe opportunities it provided for socializing, but
about seven or eight years ago, I began in earnest to
think about and study how to become a better golfer.
When playing golf, you come to realize that“patience is the key.” When starting out, it only took
one bad shot for me to feel disappointed in my game,but recently, I have learned that I can recover even if
I totally botch one hole. This knowledge is a gift thatNTT Technical Review

Front-line Researchers

comes from actually playing the game; there is a limit
to what you can learn from golf theory alone. I said
earlier that it is difficult for a researcher to keep
research-related matters out of his or her mind, but as
for me, it’s amazing that it’s only when I’m playing
golf that I am not thinking about research. Golf is agreat way to relieve stress!
Don’t become a “jack of all trades, master ofnone” when young! First, sharpen yourskills in your own research field.
—Dr. Ueda, can you leave us with some advice foryoung researchers?
I would be happy to. I have said that imagination isimportant in creating scenarios in big data analysis
and that there is a need to broaden one’s outlook andknowledge. However, there is an appropriate time forthat depending on one’s age, since a young person
who tries to do too much at the same time runs therisk of becoming a “jack of all trades, master ofnone.” When you are young, I believe it’s important
to sharpen your skills in your own research field without worrying too much about results or the usefulnessof those results. Today, the research environment isexperiencing some difficult conditions, and even universities must compete for funds, which means thateven young researchers must be adept at emphasizing
the social value of their work. Just the same, researchof a superficial nature can hardly lead to success inthe end. Since young researchers are blessed withtime and physical strength and are at the peak of theirmental abilities, doesn’t it make sense that theyshould sharpen their skills in their own research fields
while they are still young? The way to succeed is tostay focused and to “seize the bull by the horns” without getting discouraged by criticism such as being
told their kind of research is not useful. Of course,

Vol. 11 No. 11 Nov. 2013

there is also a need to maintain a sense of balance interms of research beneficial to society, but if you are
not ready to make an all-out effort to build an appreciable foundation of basic skills in an enthusiastic
manner, working only with ideas will soon reach its
limit. “Sharpen up your basic skills” is first and foremost a message for young staff involved in basic
research, but I believe that skill sharpening could also
be a beneficial approach for young staff in other fields
too, such as application and development work.

Naonori Ueda
Director of Machine Learning Data Science
Center, Senior Distinguished Scientist, NTT
Communication Science Laboratories.
He received the B.S., M.S., and Ph.D. degrees
in communication engineering from Osaka
University in 1982, 1984, and 1992, respectively.
He joined the Yokosuka Electrical Communication Laboratories of Nippon Telegraph and Telephone Public Corporation (now NTT) in 1984. In
1994, he moved to NTT CS Labs in Kyoto, where
he has been researching statistical machine learning, Bayesian statistics, and their applications to
web data mining. From 1993 to 1994, he was avisiting scholar at Purdue University, Indiana,
USA. He is a guest professor at the National Institute of Informatics and a visiting professor at
Kyoto University. He is a Fellow of the Instituteof Electronics, Information and Communication
Engineers and a member of the Information Processing Society of Japan and IEEE. He became
Senior Distinguished Scientist in April 2013 andwas appointed Director of Machine Learning
Data Science Center in July the same year, after
serving as Director of NTT Communication Science Laboratories for three years.

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data

Applications of Big Data Analytics
Technologies for Traffic and Network
Management Data––Gaining Useful
Insights from Big Data of Trafficand Network Management
Kohei Shiomoto
Abstract

The complexity of telecommunication networks continues to increase. Likewise, the complexity of
traffic carried over these networks is also increasing. A data-driven approach is a promising techniquefor handling those complexities. We have been conducting research and development (R&D) on data
analytics for traffic and network management by applying big data technologies. This article introduces
the R&D activities involving applications of big-data analytics technologies for traffic and network
management.
Keywords: big data, traffic, network management

1. Introduction
Big data technology is attracting a great deal ofattention, and many research and development(R&D) efforts related to it are under way around theworld. These include R&D efforts on data warehouseproducts and large-scale distributed data processing
platforms such as Hadoop, and those on data analytics technologies such as machine learning and data
mining. By applying these big data technologies totraffic and network management, we expect to gain
useful insights from a huge amount of operationaldata, which we could not have exploited before without such technologies, to improve traffic and network
management processes such as network planning andengineering, and network operation. Telecommunication networks are becoming increasingly complex,
and as the Internet becomes ubiquitous in its fundamental role as a social infrastructure, the complexityof traffic carried over the telecommunication networks is also increasing. We expect that a data-driven
approach will be effective in handling those complexities. We have been conducting R&D efforts ondata analytics for traffic and network management byapplying big data technologies. This article introduces the R&D activities centering on applications of
big data analytics technologies for traffic and network management.
2. Big data for network management
Various kinds of data such as traffic measurement
data, network configuration data, and data from network failure alarms are used in managing networks.
By analyzing such data we obtain information that isuseful for network planning and engineering, as well
as for network operation (Fig. 1).
Traffic measurement data include packet count andbyte count, which are measured and collected per link
and per flow by the router as a management information base (MIB) and as Netflow data. We use moredetailed information than MIB and Netflow to
NTT Technical Review

Feature Articles

Telecom network associal infrastructure

Complex traffic
Data-driven network
management
Complex network
management

Various kinds of multivendor equipment

SNS information- Twitter- blogs

Weather/temperature

Statistics- # of subscribers etc.

Data-driven network management

Network
operation

Server logs
- WWW, mail, etc.

Social events- Elections,disasters, etc.

Signaling- DNS, SIP, etc.

Traffic data
- Packet/byte counts(per-IF/per-flow)- Packet capture (PCAP)

Network planning &engineering

Failure information
Network configuration- Alarm- Topology- Syslog- Protocol settings- Trouble tickets

Cloud

NGN

Internet

Mobile

DNS: domain name system
IF: interface
NGN: Next-Generation Network
SIP: session initiation protocol
SNS: social networking service
WWW: World Wide Web

Fig. 1. Data-driven network management.

analyze traffic, which is becoming more and more
complex. We capture and analyze all of the network
data.
Network configuration data include the connectivity between network elements such as routers,
switches, and servers, as well as information on protocol settings. The connectivity between network
elements is collected from a database of the network
management system or from the configuration files of
network elements. Protocol setting information iscollected from the configuration files of network elements and captured protocol messages.
Network failure alarms include alarm messages andlog files of network elements and trouble tickets. Syslog messages are also exploited. These messages are
unstructured and in free-text format, and are specificto each vendor. The original purpose of a syslog message is for debugging. The quantity of syslog messages generated from a network element can amountto thousands of lines per day.
Vol. 11 No. 11 Nov. 2013

In addition to the above-mentioned data, big data
generated from outside the network can also be used.
Information from social networking services (SNSs)and blogs, and data that can be collected from the web
can be used for network management. Additionary,statistics on the number of subscribers, and information on weather, temperature, and social events can beused for this purpose.
3. Value of big data analytics innetwork management
3.1 Network planning and engineering
Understanding traffic characteristics is crucial for
network planning and engineering. Telephone networks have a long history, and methods for managing
them have been developed over a long period. As theInternet becomes ubiquitous in its role as a social
infrastructure, various Internet applications haveemerged, and the complexity of traffic carried overFeature Articles

the telecommunication networks continues toincrease. In particular, video traffic accounts for alarge amount of the total traffic. It is therefore necessary to understand the details of video traffic toimplement more effective network planning andengineering. Traffic characteristics will continue tochange as new applications are developed and widely
deployed. It is therefore crucial to continue measuring and analyzing traffic data. As stated previously,
we can also benefit from other data such as statistics
on subscriber numbers, weather, and temperature, as
well as social event information, which affects theway telecommunications networks are used as asocial infrastructure.
3.2 Network operation
As the role of telecommunication networks as asocial infrastructure expands, outages of these networks have a huge impact. On the other hand, telecommunication networks are generally based on
TCP/IP (transmission control protocol/Internet protocol). Various kinds of network elements such as
switches, routers, and servers are deployed in telecommunication networks, and those network elements are manufactured by various manufacturers.
Consequently, it is becoming more and more difficult
to carry out troubleshooting when network failures
and/or service outages occur.
By analyzing big data on alarm information, log
information, trouble tickets, and network configurations, we expect to be able to carry out troubleshooting tasks more quickly and easily. Syslog messages
are unstructured and in free-text format, and are vendor-dependent. The quantity of syslog messages per
day generated from networks could amount to tens of
millions of lines in large networks that include tens of
thousands of network elements. We expect to be able
to extract useful insights from big data of syslog messages. We are also looking at big data from SNSs suchas Twitter in order to extract information that will be
useful for network troubleshooting. Twitter can be
regarded as a medium for disseminating short realtime messages. We expect to be able to rapidly detect
network service outages when they occur by analyzing the big data of tweets. The examples of using
syslog and Twitter data are discussed in more detailin the next section.
4. Case studies of big data analytics innetwork management
Here, case studies of the use of big analytics inareas of network management such as network planning and engineering, and network operation arepresented.
4.1 C ase I: Network operation (failure prediction
and detection)
By analyzing a huge amount of unstructured data(syslog messages, SNS messages, etc.), we candevelop methods of detecting failures that cannot be
detected by existing network failure monitoring systems, as well as methods of analyzing the root causesof such failures. Machine learning is used to extract
useful information for network operation (Fig. 2).
We are developing methods for analyzing syslog
messages generated from network elements to findthe root cause of failure and to detect predictive information for failures [1]. A machine learning algorithm
automatically detects a set of simultaneously occurring syslog messages to understand the network event
behind them. The method is applicable to syslog messages in unstructured and vendor-dependent formats.
It helps to reduce the tedious tasks required in trouble-shooting by prioritizing and visualizing network
events.
We are also developing methods for analyzing
Twitter messages (tweets) [1]. Twitter disseminates
short messages in a real-time manner. We expect that
we can quickly detect how customers feel about network service conditions by analyzing their tweets.
We take an approach for supervised machine learningby collecting tweets that appeared at times when network outages occurred and also at times of normal
network operation as training data. We form a classifier based on a support vector machine (SVM) byusing training data. We employ the classifier in a realtime manner to detect the tweets associated with
network service outages. We quickly detected customer opinions on network service outages by analyzing their tweets as well as by monitoring existingcall center operations.
4.2 Case II: Network operation (security)
Security is becoming a major concern in network
operation areas. The use of botnets is spreading
widely, which is threatening Internet security. Blacklists are used to contain the botnets. To prevent users
from inadvertently accessing botnets, we use a blacklist of command and control (C&C) servers of thebotnets and block the communication from users to
those C&C servers. The botnets are sophisticatedenough to expand their coverage by creating andspreading new C&C servers throughout the Internet.
NTT Technical Review

Feature Articles

Syslog analytics

Machinelearning

SNS analytics

mixi
Facebook
Twitter

Possiblecausesof faults

Status andimpactof fault

Machinelearning

100 M msgs/day
Free format
Vendor specific

400 M tweets/day
Detect failure
Can’t email

Failure predictor

Can’t download

Fig. 2. Network failure detection and root cause analysis for failure prediction.

- Blacklists are used to block access to botnets.
- Botnets are spreading and becoming more sophisticated.
- Blacklists need to be maintained to combat botnet growth.
Botnets are increasing in number.

Blacklist…

Undetected

Detected

Fig. 3. Communication pattern analytics used against botnets.

To cope with these ever-expanding botnets, we needto maintain and update the blacklists of the C&C
servers (Fig. 3). We therefore analyze traffic data to
improve our coverage of the blacklists.
We have developed a method to find new unknown
C&C servers by exploiting the rule of thumb that auser’s personal computer (PC) that accesses a C&C
server of a botnet and is consequently infected byVol. 11 No. 11 Nov. 2013

malware, is likely to access other C&C servers aswell [2]. We analyze traffic data and calculate the cooccurrence score between the server communicatingwith an infected server and the already-known C&C
server communicating with the infected server. We
assume that a server with a high co-occurrence scoreis a newly discovered C&C server. We expand theblacklist by registering the newly discovered C&CFeature Articles

servers in the blacklist.
An infected user PC is under the control of the
C&C server. It can then be used to carry out security
attacks such as DDOS (distributed denial-of-service)attacks, spam mail, and information theft. We analyze
the communication patterns to find the ever-increasing C&C servers and block communications to themto contain the spreading botnets.
4.3 Case III: Network planning and engineering
The Internet is becoming ubiquitous and is playing
a fundamental role as a social infrastructure. Accordingly, numerous Internet applications have emerged,
and the complexity of traffic carried over the telecommunication networks is increasing. In particular,
video traffic accounts for a large amount of the total
traffic. We therefore need a detailed understanding ofvideo traffic to implement effective network planningand engineering. Video services are categorized as
broadcast, VoD (video-on-demand), and OTT (overthe-top). These services have a large amount of content to meet customer needs. Customer behavior isquite different from that of traditional telephone networks. Analyzing customer behavior in their viewingof video services is crucial to understand video traffic. Thus, we take into consideration concurrent
weather and temperature data and social event information to better understand video traffic [3].
We anticipate that customer behavior and their purpose in using telecommunication are constantly
changing. In addition, when broadband service isprovided by both wired and wireless access networks,
it is crucial to understand the characteristics of customer demand for effective network planning andengineering. By analyzing the statistics on the number of subscribers of broadband services, we havedeveloped a theory on S-shaped growth that enables
more accurate forecasting of future traffic demands[4].
These days, the business structure between telecom
network providers and telecom service providers is
becoming more complex. Internet service providers(ISPs) and content service providers (CSPs) collaborate in delivering content to customers. The CSP
employs a content delivery network (CDN) infrastructure for delivering content to customers using
the network infrastructure provided by the ISP. The
business objectives of the ISP and CSP are different.
The ISP executes its operation including traffic control and network planning and engineering according
to its own objectives. The CSP executes its operation
including content delivery control and content alloca

tion control according to its own objectives. It is
crucial to understand their objectives and how theyinteract with each other in performing their operations when we execute network planning and engineering operations [5].
5. Platforms for data analytics
5.1 Platform for R&D
We are building a data analytics platform to collect,
store, and analyze various kinds of traffic and network management data in order to conduct R&D
efforts related to the application of big data analytics
technologies for traffic and network management. We
are using high-end commodity servers and a software
framework for data-intensive distributed applications
such as Hadoop to build our data analytics platform.
We are developing algorithms for data analytics
using high-end servers and we are making softwaretools that implement the data analytics algorithms wedeveloped using machine learning, data-mining, andstatistical analysis.
The machine learning algorithms we are using todevelop the data analytics algorithms include nonnegative matrix factorization (NMF) [6], [7] and support vector machine (SVM) [8] to find hidden rules
among big data and to make classifiers using training
data. We are developing data analytics algorithms for
unstructured data such as Twitter data and syslog datausing natural language processing techniques including Japanese language morphological analysis andsyntactic analysis techniques based on word-classification and clustering methods. We have implemented
a real-time Twitter analytic tool using the online
machine learning framework Jubatus [9] for pre-processing and an SVM type machine learning algorithm.
Traffic is becoming more and more complex. We
need to capture the entire data of packets and analyze
them. The amount of data can be quite huge even fora short period. To cope with this, we use a software
framework for data-intensive distributed applications
such as Hadoop. We are implementing our algorithms
using the MapReduce programming model for parallel processing of huge data.
5.2 Platform for data analytics
We are conducting our R&D efforts by applying
big-data analytics technologies to traffic and network
management data.
Real data is crucial for these R&D efforts. We collaborate with business units including network
NTT Technical Review

Feature Articles

planning and engineering, and network operations in
conducting R&D efforts. Traffic and network management data are measured and collected by the business units. We analyze those data in the R&D labs.
We verify the analytic results through discussions
with operators and engineers in the business units.
We establish analytics methodologies through theabove-mentioned R&D process and transfer them tothe business units so that they can apply those methodologies in their operation processes. The R&D
outcome, in the form of technical documents, operation manuals, and software tools and engines built
into the operation systems, thus contributes to thebusiness process innovations achieved in the business
units.
R&D efforts for big data analytics technologies
require cross-domain expertise including databasetechnologies such as Hadoop and data warehousing,and analytics technologies such as machine learning,data mining, and statistical analysis. We are collaborating on these technologies with other departments
in NTT labs to conduct our R&D efforts.
We expect that our data-driven approach using big
data analytics technologies for traffic and network
management data will continue to evolve and willcontribute to producing a wide range of R&D outcomes. We will continue collaborating with our business units and other R&D departments to conduct our
R&D efforts that exploit cutting-edge technologies
developed in our R&D labs.
References
[1]

[2]

[3]

[4]

[5]

T. Kimura, K. Takeshita, T. Toyono, M. Yokota, K. Nishimatsu, and T.
Mori, “Network Failure Detection and Diagnosis by Analyzing Syslog and SNS Data: Applying Big Data Analysis to Network Operations,” NTT Technical Review, Vol. 11, No. 11, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
11fa4.html
K. Sato, K. Ishibashi, T. Toyono, and N. Miyake, “Extending Black
Domain Name List by Using Co-occurrence Relation Between DNSQueries,” Proc. of the 3rd USENIX Workshop on Large-Scale
Exploits and Emergent Threats (LEET ’10), San Jose, CA, USA, April
2010.
S. Kouno and H. Hasegawa, “Traffic Management of Video Streaming
Services,” NTT Technical Review, Vol. 11, No. 11, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
11fa3.html
S. Shimogawa, K. Takeshita, and K. Nishimatsu, “Analyzing Macroscopic Demand Structure Based on New Model of S-shaped Growth,”
NTT Technical Review, Vol. 11, No. 11, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
11fa5.html
K. Ishibashi, S. Harada, and S. Kamei, “Analyzing Internet Traffic
Structure through Big Data Technology,” NTT Technical Review, Vol.
11, No. 11, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
11fa2.html

Vol. 11 No. 11 Nov. 2013

[6]
[7]

[8]
[9]

D. D. Lee and H. S. Seung, “Learning the Parts of Objects by NonNegative Matrix Factorization,” Nature, Vol. 401, pp. 788–791,
1999.
K. Ishiguro and K. Takeuchi, “Extracting Essential Structure from
Data,” NTT Technical Review, Vol. 10, No. 11, 2012.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2012
11fa2.html
C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, Vol. 20, No. 3, pp. 273–297, 1995.
K. Horikawa, Y. Kitayama, S. Oda, H. Kumazaki, J. Han, H. Makino,
M. Ishii, K. Aoya, M. Luo, and S. Uchikawa, “Jubatus in Action:
Report on Realtime Big Data Analysis by Jubatus,” NTT Technical
Review, Vol. 10, No. 12, 2012.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2012
12fa5.html

Kohei Shiomoto

Senior Manager of Communication & Traffic
Service Quality Project, NTT Network Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees
in information and computer sciences from
Osaka University in 1987, 1989, and 1998,
respectively. He joined NTT in 1989 and engaged
in R&D of ATM traffic control and ATM switching system architecture design. During 1996–
1997, he was engaged in research on high-speed
networking as a Visiting Scholar at Washington
University in St. Louis, MO, USA. During
1997–2001, he directed architecture design forthe high-speed IP/MPLS label switching routerresearch project at NTT Network Service Systems Laboratories. He was engaged in research
involving photonic IP router design and routing
algorithms, and GMPLS routing and signaling
standardization, first at NTT Network Innovation
Laboratories and then at NTT Network Service
Systems Laboratories from July 2001 to June
2011. From April 2006 to June 2011, he led the
IP Optical Networking Research Group in NTT
Network Service Systems Laboratories. He wasinvolved in standardization of GMPLS in the
IETF. He also led the traffic engineering researchgroup in NTT Service Integration Laboratories
from July 2011 to June 2012. Since July 2012, he
has also been leading the Communication &Traffic Service Quality Project of NTT Network
Technology Laboratories. He is a fellow of
IEICE, a member of IEEE and the Association
for Computing Machinery. He has been the Chairof various committees of the Asia-Pacific Boardof the IEEE Communications Society. He was
the Secretary for International Relations of theCommunications Society of IEICE from June
2003 to May 2005. He has been involved in theorganization of several international conferences
including MPLS, iPOP, and WTC. He received
the Young Engineer Award from IEICE in 1995
and the Switching System Research Award fromIEICE in 1995 and 2000.

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data

Analyzing Internet Traffic Structure
through Big Data Technology
Keisuke Ishibashi, Shigeaki Harada, and Satoshi Kamei
Abstract

The volume of Internet traffic consisting of content delivered by large content providers continues toincrease. Moreover, the traffic control measures used by those content providers make the dynamics ofInternet traffic more complex. In this article, we introduce a method of analyzing the complex dynamics
of Internet traffic and present some results obtained after applying the method.
Keywords: Internet, content delivery, traffic

1. Introduction
The volume of Internet traffic has long been
increasing exponentially. It was reported recently that
not only is the volume increasing, but the composition is also drastically changing. The major portion of
traffic is changing from P2P (peer-to-peer) file sharing to video content delivery from large content service providers (CSPs) [1]. This has led to a change inthe structure of traffic exchanges among autonomous
systems (ASes), specifically, between CSPs andInternet service providers (ISPs) [2]. Those CSPs are
deploying their cache servers inside the ISP networks
or establishing direct peering to ISPs to reduce theresponse time for content retrieval as well as the transit cost if they are otherwise required to pay a fee tothe transit providers.
In addition to the changing trend in the structure ofInternet traffic delivery, these CSPs change the allocations of their content servers for users’ content
requests. This leads to short-term traffic fluctuationsin the ISP networks. These traffic fluctuations and thechanging trend in traffic delivery bring new challenges in bandwidth provisioning and traffic management for ISPs. Long-term changes in the content
delivery structure lead to changes in peering strategy,
and short-term fluctuations make it difficult to predict
and manage traffic.
We describe here how to measure and analyze thesechanges and fluctuations in the Internet traffic deliv

ery structure. Specifically, we decompose trafficdelivery into users’ traffic demands, CSPs’ content
supply, and ISPs’ content delivery. By decomposing
the traffic delivery structure into these three components, we can identify the changes in the structure andthe root causes for the changes, making it possible topredict such changes and develop countermeasures.
2. Traffic decomposition and measurement
The decomposition of network traffic into traffic
demands of users, content supply by CSPs, and traffic
distribution by ISPs is shown in Fig. 1. User demands
are mapped to network traffic by control techniquesapplied by both CSPs and ISPs.
2.1 User traffic demands
Content delivery traffic originates with the users’
traffic demands. Here, traffic demands indicate users’
requests for content, for example, video, to CSPs.
Those demands are calculated by monitoring users’
access logs that are captured by network monitoring
agents [3].
2.2 CSP content supply
CSPs dynamically allocate a content distribution
server for each user content request in order to optimize load balancing and the quality of content delivery. Normally, the nearest server to the user is allocated, but in some cases, other servers may be
NTT Technical Review

Feature Articles

Server selection
control

CSP

CDN
Path control

ISP

CSP cache

User

CDN: content delivery network
Networktraffic

User traffic
demands forcontent=

×

Mapping ofcontent toserver×

Mapping
server topath

Fig. 1. Decomposition of network traffic into user demand, CSPs’ content supply, and ISPs’ traffic distribution.

Overseas

CDNs
Download traffic

BGP
collector

Japan
Passivemonitoring

Cache server

ISPs
DNS server

Active monitoring(emulate user’s request)

Measurement
infrastructure

Measurement
probesUser

Web server

Cash server

BGP: border gateway protocol

Fig. 2. Methods of allocating servers.

allocated based on the status of server load, network
congestion, or server failures. These changes in allocation cause short-term fluctuations in network traffic.
Server allocation methods are roughly divided into
Vol. 11 No. 11 Nov. 2013

two categories: DNS (domain name system) based
methods and web server based methods (Fig. 2). In
the former, a content distribution server is assigned toa user request as an Internet protocol (IP) address ina DNS response for the query from the user. In thisFeature Articles

Disks indicate ASes. (blue:in Japan; orange: overseas)

Disk radius indicatesthe amount of outgoingtraffic from the AS.

Fig. 3. Visualization of traffic structure.

allocation, the DNS response IP address is determined based on the source IP address of the DNS
query, which is normally not the user’s IP address but
a DNS cache server’s IP address. By contrast, in thelatter type, when a user accesses a web server to viewcontent, the server dynamically allocates a server forthe content request and returns an HTML (hypertext
markup language) page with a URI (uniform resource
identifier) that includes the server.
These allocations can be measured by using passiveor active monitoring techniques, specifically, by passively monitoring the network traffic or by actively
accessing the content. By monitoring DNS traffic, we
can observe how a user’s request for content ismapped to an IP address of the server. This passive
monitoring is advantageous in that it allows all user
requests to be monitored inside the monitored network. The advantage of the latter monitoring technique is that it can monitor the mapping of multiple
networks by sending emulated user requests fromoutside the network.
2.3 ISP traffic delivery
Once the server for the content request is determined, then the path between the user and the serveris determined by ISP routing control. Normally, thiscontrol is not limited to a single ISP, but spans multiple ISPs because CSPs, which have content, andISPs, which have users, are different autonomous
systems (ASes). ASes control the routing to optimizethe quality of content delivery and also to minimize
the transit costs that depend on the relationshipbetween ASes.
Monitoring of the routing controls is also measured
by passively monitoring routing control messages
such as BGP (Border Gateway Protocol) messages[4], or by actively sending probe packets for routing
using a tool such as traceroute. The advantages ofboth types of monitoring are the same as for theserver allocation, in that the former type can monitor
all traffic inside the network, whereas the latter canmonitor the statuses of multiple networks. Inferring
traffic exchanges in a network from outside the network is said to be difficult [5], but by decomposing
the user requests, server selection, and routing control, we can infer traffic exchanges from outside the
network, as described in the next section. If we canassume that the popularity of the content that users
request is roughly the same among the ISPs, then we
can infer the network traffic exchanges by monitoring
the server selection and routing control from outside
the network.
3. Measurement results
We applied the active access measurement method
explained in the previous section to analyze traffic.
Examples of visualizing HTTP (hypertext transfer
protocol) download traffic for two networks using
this method are shown in Fig. 3. Each node represents
NTT Technical Review

Feature Articles

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%

Weekend

22:00:00

20:00:00

18:00:00

16:00:00

14:00:00

12:00:00

10:00:00

8:00:00

6:00:00

2:00:00

4:00:00

0:00:00

22:00:00

20:00:00

18:00:00

16:00:00

12:00:00

14:00:00

8:00:00

10:00:00

6:00:00

4:00:00

0:00:00

2:00:00

0%

Weekday: Accesses allocated a server in Japan: Accesses allocated a server overseas

Fig. 4. Results of observing traffic over two-day period from three measurement points in Japan.

an AS, and the edge between two nodes represents the
traffic exchange between the ASes. The disks indicate
larger nodes; the radius of the disk indicates thevolume of traffic from that AS. In these visualizations, users’ traffic demands are assumed to be thesame for the two networks, but because the server
selections and routing controls are different, theresulting traffic structures are significantly different.
Specifically, we confirmed that the network on theright has a content cache server of CSPs inside the
network and that the server is allocated for almost alluser content requests; thus, the traffic for the content
requests flows inside the AS. This observation corresponds to a recent report on the trend of Internet traffic structure changes [1].
In addition to the change in the structure of traffic,
we also observed short-term fluctuations in CSP
server allocation. The results of observing traffic
from three measurement points in Japan are shown in
Fig. 4. The orange bars indicate the ratio of accesses
that were allocated a server in Japan, and the green
bars indicate the accesses allocated servers fromoverseas. These changes affect the bandwidth provisioning of links connected to other ASes.
4. Conclusion

Fig. 5. Visualization of connection status.

ture for network data measurement, analysis, andvisualization that is based on the Internet traffic structure analysis method described in this article. The
measurement infrastructure is built on a network thathas BGP transit connections with multiple ISPs. We
are also developing a visualization tool that shows theinter-AS connection status, as shown in Fig. 5.

We, as an NTT Communications advanced IP
architecture center, are now developing an infrastrucVol. 11 No. 11 Nov. 2013



Feature Articles

References[1][2]

K. Cho, “Broadband Traffic Report: Traffic Trends over the Past
Year,” The Internet Infrastructure Review, Vol. 16, pp. 33–37, August
2012.
C. Labovitz, S. I. Johnson, D. McPherson, J. Oberheide, and F.
Jahanian, “Internet Inter-Domain Traffic,” Proc. of the ACM SIGCOMM’10, pp. 75–86, New Delhi, India, 2010.

Keisuke Ishibashi

Senior Research Engineer, Supervisor, NTT
Network Technology Laboratories.
He received the B.S. and M.S. degrees in mathematics from Tohoku University, Miyagi, in 1993
and 1995, respectively, and the Ph.D. degree in
information science and technology from the
University of Tokyo in 2005. Since joining NTT
in 1995, he has been engaged in research on traffic issues in computer communication networks.
He received the Young Researcher’s Award fromIEICE in 2002, the Information Network
Research Award in 2002 and 2010, and the Internet Architecture Research Award in 2009. He is amember of IEEE, the Institute of Electronics,
Information and Communication Engineers, andthe Operations Research Society of Japan.

[3]
[4]

[5]

Alexa. http://www.alexa.com/
A. Kobayashi, S. Kashima, H. Kurakami, and K. Ishibashi, “Traffic
Monitoring System Based on Correlation between BGP Messages andTraffic Data,” IEICE Trans. Commun., Vol. E94-B, No. 9, pp. 2532–
2542, Sep. 2011.
H. Chang, S. Jamin, Z. M. Mao, and W. Willinger, “An Empirical
Approach to Modeling Inter-AS Traffic Matrices,” Proc. of the 5th
ACM SIGCOMM Conference on Internet Measurement, p. 12, Philadelphia, PA, USA, 2005.

Satoshi Kamei

Manager, NTT Communications.
He received the B.S. and M.S. degrees inapplied mathematics and physics from Kyoto
University in 1997 and 1999, respectively. In
1999, he joined NTT Service Integration Laboratories, where he engaged in measuring and analyzing overlay network traffic. He moved to NTT
Communications in 2012.

Shigeaki Harada

Research Engineer, NTT Network Technology
Laboratories.
He received the B.E., M.E., and D.E. degrees
from Tohoku University, Miyagi, in 2001, 2003,
and 2006, respectively. In 2006, he joined NTT
Service Integration Laboratories. From 2009 to
2012, he was with the R&D Center, NTT WEST.
In 2012, he moved to NTT Network Technology
Laboratories. Since joining NTT, he has been
engaged in research on traffic measurement andanalysis of IP networks.



NTT Technical Review

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data

Traffic Management ofVideo Streaming Services
Shouji Kouno and Haruhisa Hasegawa
Abstract

The use of video streaming services is becoming more widespread, and as a consequence, the traffic
volume of these services is rapidly increasing over the broadband communication network. It sometimes
happens as a rare event that many users will start using video streaming services almost simultaneously.
This results in an explosive increase in the volume of traffic, which is observed in the network data. Users
strongly desire video streaming services that provide clear and high-resolution images for the entire
duration of the programs they are viewing. Thus, special attention must be paid to these services in thetask of traffic management. In this article, we characterize the variation in the traffic volume of video
streaming services based on the viewing activities of users, and we explain what knowledge is necessary
to manage the traffic.
Keywords: traffic management, VOD, traffic volume

1. Introduction
1.1 T raffic characteristics of video streaming
services
In July 2011, Japan shifted to a digital terrestrial
television (TV) broadcasting system. Around thattime, the traffic volume of video streaming services
began to increase rapidly over the broadband communication network. Video streaming services can beclassified into Internet protocol (IP) multi-channel
services and video on demand (VOD) services. With
the former, a constant traffic volume can be maintained regardless of the number of users of a service
because of the utilization of IP multicast. By contrast,
the latter type increases the traffic volume in proportion to the number of users of a service because it
delivers video content data according to the requests
from users who want to view the video content.
Therefore, in managing traffic on a broadband communication network, it is very important to accurately
predict the traffic volume of the video streaming service and to dimension the amount of equipment necessary for offering the services. In this article, theterm video streaming service indicates a VOD service.


1.2 Spikes
A time series of the traffic volume of video streaming services during 2010–2011 is shown in Fig. 1.
The Y-axis shows the daily maximam value of thetraffic volume of the service. The daily maximam
traffic volume per subscriber is shown by the Z-axis.
Each axis is normalized by the value on January 1,
2010. The sudden peaks in the traffic can clearly be
distinguished. We call these peaks spikes. We compared video streaming services with conventional
communication services, for example, phone andInternet access services, and found that video streaming services have distinguishable features [1]. That is,their spikes have a special frequency and a relatively
longer length. Here, we begin to explain the trafficcharacteristics and the background mechanism ofvideo streaming services.
The traffic of a video streaming service in July2010 is plotted in Fig. 2. The X-axis indicates a range
from 6:00 a.m. to 5:00 a.m. the following day. Before
explaining the spiky increases, we first explain thedaily characteristics of the traffic volume of the video
streaming service. The traffic volume is larger onweekends than it is on weekdays, so it is understoodthat this service is mainly used by home users rather
NTT Technical Review

15.002.50

Daily maximal value of traffic volume (normalized)

14.00
13.00
12.00

2.00

11.00
10.00
9.00

1.50

8.00
7.00
6.00

1.00

5.00
4.00

Daily maximum (left)

3.00

Daily maximum per subscriber (right)

0.50

2.00
1.00
0.00

Daily maximal value of traffic volume per subscriber (normalized)

Feature Articles

0.00

2010/01/01 2010/04/01 2010/07/01 2010/10/01 2011/01/01 2011/04/01 2011/07/01 2011/10/01 2012/01/01* The value of both the Y and Z axes are normalized: The value of January 1, 2010 is 1.00.

Fig. 1. Changes in VOD traffic volume.

June 11, 2013

Weekends

Weekdays

67

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0

1

2

3

4

5

Time

Fig. 2. Impact of news reports on Upper House election on VOD traffic.

than by businesses.
We can see that the busiest hour of the day, whichrefers to the period with the largest daily traffic volVol. 11 No. 11 Nov. 2013

ume of the service, is usually from 22:00 to 23:00
regardless of whether it is a weekday or weekend day.
An exception is seen at around 16:00 on weekends,
Feature Articles

June 13 (Sun), 2010
Weekends

vs. Cameroon
Weekdays
vs. Netherlands
vs.
Paraguay
対パラグアイvs. Denmark
対デンマーク6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0 1 2 3 4 5Time

Fig. 3. Impact of FIFA World Cup on VOD traffic.

when the daily traffic volume reaches the highest
peak.
Some odd phenomena in the pattern of traffic canbe seen. After the peaks in the early evening on weekends, the traffic volume begins to decrease until 19:00
or 20:00, then starts increasing until 23:00. This looks
like an unnatural trajectory.
These temporary decreases are special characteristics of the traffic volume of video streaming services.
1.2.1 Illustrative case 1
Let us introduce some illustrative cases of spikes.
At 20:00 on Sunday, July 11, 2010, the traffic volume of video streaming services began to increase
suddenly. Note that the traffic volume had shown
almost the same volume relative to ordinary weekends until 19:00. In fact, that day was the day of theHouse of Councillors election in Japan. At exactly
20:00, almost all Japanese TV stations simultaneously started broadcasting a special news report on
the election results.
TV stations usually broadcast TV programs in competition with other stations. There are various categories of programs, e.g., dramas, movies, music programs, live sporting events, educational programs,
entertainment programs, cartoons, news, and special
reports. Therefore, it is possible for people to choosea favorite TV program from the available TV programs whenever they want to watch them.
On this occasion, however, people could not freely
select their TV programs because all the TV stations


were broadcasting the same content—the election
news report.
We can easily imagine that at this time, some people who were unconcerned with the election results,
for example, children, could not find a TV program
they wanted to watch, so they turned to video streaming services instead.
From this illustrative case, a relationship appears
between the traffic volume of video streaming services and TV programs being broadcast at that time.
1.2.2 Illustrative case 2
Now we explain a reverse example to support therelationship of illustrative case 1. A 24-hour graph ofthe traffic volume of a video streaming service inJune 2010 is shown in Fig. 3. During that month, theFIFA (Fédération Internationale de Football Association) World Cup games were being held in South
Africa.
When the matches of the Japanese national team
were broadcast live on TV, the traffic volume of video
streaming services was substantially reduced in comparison with the same time on other days that monthwhen the Japanese team was not playing.
The presumption is that even heavy users of video
streaming services were refraining from using
streaming services at that time so that they could
watch the match live on TV and cheer for the Japanese national team.
As expected, we can see that there is a strongly
competitive relationship between TV broadcasting
NTT Technical Review

Feature Articles

5
Awake-at-home rating
TV viewing rating
VOD traffic volume per subscriber

80

4

60

3

40

2

20

1

0

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 0

1

2

3

4

5

VOD traffic volume per subscriber (normalized)

Awake-at-home rating & TV viewing rating (%)

100

0

Time
* TV viewing rating was provided by Video Research Ltd.

Fig. 4. Fluctuation of awake-at-home rating, TV viewing rating, and VOD traffic volume per subscriber over 24 hours(6 a.m. Sunday through 5 a.m. Monday).

and video streaming services.
2. Human life activities
Here, let us explain the characteristics of the traffic
volume of video streaming services in connection
with people’s activities in their daily lives. Our
research is based on the Survey on Time Use and Leisure Activities (2006) conducted by the Statistics
Bureau, Ministry of Internal Affairs and Communications [2].
This survey aims to obtain comprehensive data on
the daily patterns of time allocation and on leisureactivities. The survey was first carried out in 1976 andsince then has been conducted every five years. In thesurvey, the enumerators deliver questionnaires toeach household to be surveyed, collect the completed
questionnaires, and interview the households as necessary. The questionnaires are completed by household members 10 years old and over or by the head of
the household. The participants also answer questions from the enumerators. To grasp the distributionof time use during a 24-hour period, each household
member is requested to classify his/her own activities
into 20 categories (1. Sleep, 2. Personal care, 3.
Meals, 4. Commuting to and from school or work, 5.
Vol. 11 No. 11 Nov. 2013

Work, 6. Schoolwork, 7. Housework, 8. Caring or
nursing, 9. Child care, 10. Shopping, 11. Moving(excluding commuting), 12. Watching TV, listening
to the radio, reading newspapers or magazines, 13.
Rest and relaxation, 14. Study and research (excluding schoolwork), 15. Hobbies and amusements, 16.
Sports, 17. Volunteer and social activities, 18. Social
life, 19. Medical examinations or treatment, 20. Other
activities).
For these activities, we define an awake-at-homerating as the sum of the ratings of the activities being
done at home except for category 1, Sleep.
A 24-hour time series of an awake-at-home rating,
TV viewing rating [3], and traffic volume per subscriber of video streaming services of Sundays average from 6 a.m. to 5 a.m. the following day is plottedin Fig. 4. We can see that the TV viewing rating has
almost the same shape as the awake-at-home rating.
In fact, when many people come home, one of thefirst things they do is turn on the TV. Then they continue watching TV, or just leave the TV on, while they
are eating dinner or doing housework. Finally, theyturn the TV off just before going to bed. Such a custom is considered to be normal, so this relation
between the two ratings is easily understood. In
addition to the TV viewing rating, the awake-at-homeFeature Articles

rating also peaks at around 20:00.
In the field of TV broadcasting, this time occurs
during the period called the prime time in Japan(19:00–22:00). TV broadcasting companies produce
TV programs such as dramas or entertainment programs by assembling casts of popular TV celebrities
and actors/actresses in order to achieve a higher TV
viewing rating than others. These broadcasting companies are always strongly motivated to broadcast TV
programs that are expected to achieve higher TV
viewing ratings leading into the prime time because
the aim of their business is to obtain higher advertising revenues.
2.1 Relation to TV broadcasting
Here, let us consider the relation between video
streaming services and TV broadcasting. The traffic
volume of the former has a characteristic in which it
temporarily decreases at around 20:00. As mentioned
previously, 20:00 is the prime time in the TV broadcasting business when all TV channels are broadcasting programs that are more popular than video
streaming services. This is the same as the case of theFIFA World Cup.
In Fig. 4, it is shown that the traffic volume of video
streaming services increases on Sunday afternoons.
The quality of TV content being broadcast at this timeis perceived to be lower than at other periods, somany subscribers feel that video content is moreinteresting than regular TV programs.
In fact, at this period of time, many TV channels are
rebroadcasting previously aired TV programs.
2.2 Relation to awake-at-home rating
Here, we discuss the relation between video streaming services and the awake-at-home rating. The
attractiveness of TV programs is often assumed to be
at a constant level throughout the day. In this situation, it is considered a natural trend that if the awakeat-home rating increases, then the utilization andtraffic volume of video streaming services also
increase. What would cause the awake-at-home rating in the daytime to be the highest on weekends, as
shown in Fig. 3? It is clear from this figure that theday the traffic volume of the video streaming service
was highest was June 13th, which was a Sunday.
An image of a rain cloud at 12:00 on that day isshown in Fig. 5 [4].
The rain cloud was positioned over the middle of
Honshu Island, and it brought rain to the western partof Japan in the morning. Then this cloud brought rain
to the eastern part of Japan in the afternoon [5]. WeJune 13, 2010
12:00

Fig. 5. S atellite image of rain cloud at noon (12:00), June
13, 2010.

presume this is the reason June 13th had the highest
traffic volume among weekdays and weekend days
that June.
Such weather triggers of video streaming services
occur not only with rain but also with decreasing
temperatures. Both typhoons in summer and cold
spells in winter produce this effect.
It is well understood that weather conditions leadsubscribers to stay indoors and that an increasing
awake-at-home rating generates demand for video
streaming services.
3. Future prospects in traffic research
In this article, we introduced the traffic characteristics of video streaming services and explained why
the traffic variation has the shape described above.
Furthermore, it has been derived theoretically andverified from data that the traffic volume of video
streaming services per subscriber obeys a lognormal
distribution. The techniques described here are now
ready to be investigated as on-site applications of theoperating companies of the NTT Group.
It is expected that a great variety of services will beoffered over the broadband communication network
in future. It is therefore important to analyze andunderstand the unknown traffic characteristics ofsuch services in order to manage the traffic appropriately.
We are beginning to apply the big data approach inorder to acquire more detailed knowledge of traffic.
Therefore, we are utilizing not only the traffic datathat are measured inside networks but are also acquiring large amounts of information that exist beyond
NTT Technical Review

Feature Articles

networks such as the weather information mentioned
above, TV programs, TV viewing ratings, and awakeat-home ratings.
References[1]

H. Hasegawa, S. Kouno, A. Shiozu, M. Sasaki, and S. Shimogawa,“Predictive Network Traffic Engineering for Streaming Video Ser-

Shouji Kouno

Senior Research Engineer, Communication
Traffic & Service Quality Project, NTT Network
Technology Laboratories.
He received the B.E. and M.E. degrees in information science from Tokyo Institute of Technology in 1991 and 1993, respectively. He joined
NTT in 1993 and has been researching trafficengineering and control in multimedia networks.
He is a member of the Institute of Electronics,
Information and Communication Engineers(IEICE) and the Operations Research Society ofJapan.

Vol. 11 No. 11 Nov. 2013

[2]
[3]
[4]
[5]

vice,” Proc. of the IFIP/IEEE International Symposium on Integrated
Network Management, Ghent, Belgium, 2013.
The Statistics Bureau, Ministry of Internal Affairs and Communications.
http://www.stat.go.jp/english/index.htm
Video Research Ltd. http://www.videor.co.jp/eng/index.html
Weather forecast (in Japanese). http://tenki.jp/past/
Japan Meteorological Agency.
http://www.jma.go.jp/jma/indexe.html

Haruhisa Hasegawa

Senior Research Engineer, Supervisor, Group
Leader, Traffic Solution Group, Communication
Traffic & Service Quality Project, NTT Network
Technology Laboratories.
He received the B.E., M.E., and Ph.D. degrees
in information and computer science fromWaseda University, Tokyo, in 1988, 1990, and2001, respectively. He joined NTT in 1990 andhas been researching traffic engineering andmanagement in telecommunications networks.
From 2000 to 2003, he was engaged in network
management at the network operation center of
NTT EAST. He is a member of IEICE and theIEEE Communications Society.

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data

Network Failure Detection andDiagnosis by Analyzing Syslog and
SNS Data: Applying Big Data
Analysis to Network Operations
Tatsuaki Kimura, Kei Takeshita, Tsuyoshi Toyono,
Masahiro Yokota, Ken Nishimatsu, and Tatsuya Mori
Abstract

We introduce two big data analysis methods for diagnosing the causes of network failures and for
detecting network failures early. Syslogs contain log data generated by the system. We analyzed syslogs
and succeeded in detecting the cause of a network failure by automatically learning over 100 million logs
without needing any previous knowledge of log data. Analysis of the data of a social networking service(namely, Twitter) enabled us to detect possible network failures by extracting network-failure related
tweets, which account for less than 1% of all tweets, in real time and with high accuracy.
Keywords: big data, syslog, network failure detection

1. Introduction

2. Log data analysis

Internet protocol (IP) networks consist of manykinds of equipment from different vendors. These
networks are becoming much more complex becauseof the increasing demand for new and different applications. Additionally, many of these applications areprovided by multiple network operators and devices,and this makes it very difficult to diagnose network
failures when they occur. Consequently, it is very
important to develop methods to efficiently detect
network failures and diagnose their causes.
In this article, we introduce two methods for analyzing data from syslogs and from a social networking service (SNS) to achieve early network failure
detection and to diagnose the cause of the network
failure that current operating methods cannot
address.

Network operators monitor various kinds of information such as trap information from network elements, network traffic, CPU (central processingunit)/memory utility data, and syslog data. In particular, the syslog data of network elements such as routers, switches, and RADIUS (Remote Access Dial In
User Service) servers include detailed and precise
information for troubleshooting and monitoring thehealth of networks when configurations change.
However, analyzing log data has become very difficult for the following reasons:(i)	There are various types of logs, which list messages with low or high severity. In addition, theincrease in the number of network elements
means there is a massive volume of complex log
data, and it is therefore necessary to extract
information accurately and efficiently in orderto carry out troubleshooting and preventive
maintenance.



NTT Technical Review

Feature Articles

Template
extraction

Featureextraction

Log grouping

Log messages

Visualization

Log templates

Log groups

Fig. 1. Flowchart for visualization of logs.

<189> security telnet connection 15720 with 10.7.11.11 broken<189> security telnet connection 18340 with 10.8.9.123 broken<189> security telnet connection 12340 with 10.8.2.12 broken<189> security telnet connection 12345 with 10.1.1.1 broken<189> security telnet connection * with * broken

Fig. 2. Conceptual image of log template extraction.

(ii)	The log format depends on each vendor or service. Thus, understanding the meaning of eachlog message requires deep domain knowledgeof each format.
To overcome these problems, we have developed atechnique to analyze syslogs that involves automatically extracting the relationships or abnormalities
from log messages using machine-learning methods
without relying on any domain knowledge about theformat or the vendor of log data (Fig. 1). This analysis technique consists of four steps: log template
extraction, log feature extraction, log grouping, andvisualization of abnormal events.
2.1 Log template extraction
Log messages contain various parameters such as
IP address, host name, and PID (process identification). Because parameter words are very rare, log
messages with unique parameters may never appear
twice even though the events the messages signify are
the same. Therefore, we automatically extract a primary template from all log messages based on theobservation that parameter words appear infrequently
in comparison with template words in the other positions (Fig. 2). The log template enables us to easily
correlate log messages.

Vol. 11 No. 11 Nov. 2013

2.2 Feature extraction
As mentioned before, the vendor’s severity of a log
message is not necessarily reliable because it is notdirectly related to the actual network abnormality.
Therefore, we need to quantify the abnormality andnormality of logs without considering the severity ofthe message and without requiring any domain
knowledge. For example, firewall logs and link down/
up logs related to users’ connect/disconnect events
contain very common messages and can be considered. Also, the logs generated by cron* jobs or in
regular monitoring are not as frequent but are generated periodically on a daily basis. Therefore we
define the frequency and periodicity features for log
messages.
2.3 Log grouping
Typically, network operators do not use a one-line
log message, but rather, a group of logs. For example,
a router reboot event induces multiple logs, whichindicates that various processes start at the same time.
Thus, we need to group them in terms of their cooccurrence. Grouping logs reduces the volume oflogs and helps operators make sense of the logs. For*

A time-based job scheduler used in Unix-like computer operating
systemsFeature Articles

2012-1-1T00:00:00 %TRACKING-5-STATE: 1 interface Fa0/0 line-protocol Up->Down
2012-1-1T00:00:00 %LINK-3-UPDOWN: Interface FastEthernet 0/9, changed state to down
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T01:11:00 msg [100]: STP: VLAN 1 Port 38 STP State -> DISABLED (PortDown)
2012-1-1T01:11:00 msg [101]: System: Interface ethernet 38, state down
2012-1-1T03:00:00 msg [200] : STP: VLAN 100 Port 22 STP State -> DISABLED (PortDown)
2012-1-1T03:00:00 msg [201] : System: Interface ethernet 22, state down
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T10:30:00 System: Interface ethernet 1, state down
2012-1-1T10:30:00 System: Interface ethernet 1, state up
2012-1-1T10:30:00 System: Interface ethernet 2, state down
2012-1-1T12:00:00 init: alarm-control (PID 111) terminate signal sent
2012-1-1T12:00:00 init: bslockd (PID 124 ) terminate signal sent
2012-1-1T12:00:00 init: ce-l2tp-service (PID 123 ) terminate signal sent
2012-1-1T12:00:00 init: chassis-control (PID 1111 ) terminate signal sent
2012-1-1T12:00:00 init: class-of-service (PID 11112) terminate signal sent
2012-1-1T12:00:00 init: craft-control (PID 111) terminate signal sent
2012-1-1T12:00:00 init: database-replication (PID 2718932 ) terminate signal sent
2012-1-1T12:00:00 init: diameter-service (PID 2993 ) terminate signal sent
2012-1-1T12:00:00 init: disk-monitoring (PID 7082 ) terminate signal sent?
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T15:45:10 msg [200] : STP: VLAN 100 Port 22 STP State -> DISABLED (PortDown)
2012-1-1T15:45:10 msg [201] : System: Interface ethernet 22, state down
2012-1-1T16:12:40 System: Interface ethernet 1, state down
2012-1-1T16:12:40 System: Interface ethernet 1, state up
2012-1-1T16:12:40 System: Interface ethernet 2, state down
2012-1-1T20:30:00 init: alarm-control (PID 111) terminate signal sent
2012-1-1T20:30:00 init: bslockd (PID 124 ) terminate signal sent
2012-1-1T20:30:00 init: ce-l2tp-service (PID 123 ) terminate signal sent
2012-1-1T20:30:00 init: chassis-control (PID 1111 ) terminate signal sent
2012-1-1T20:30:00 init: class-of-service (PID 11112) terminate signal sent

2012-1-1T00:00:00 %TRACKING-5-STATE: 1 interface Fa0/0 line-protocol Up->Down
2012-1-1T00:00:00 %LINK-3-UPDOWN: Interface FastEthernet 0/9, changed state to down
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T01:11:00 msg [100]: STP: VLAN 1 Port 38 STP State -> DISABLED (PortDown)
2012-1-1T01:11:00 msg [101]: System: Interface ethernet 38, state down
2012-1-1T03:00:00 msg [200] : STP: VLAN 100 Port 22 STP State -> DISABLED (PortDown)
2012-1-1T03:00:00 msg [201] : System: Interface ethernet 22, state down
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T10:30:00 System: Interface ethernet 1, state down
2012-1-1T10:30:00 System: Interface ethernet 1, state up
2012-1-1T10:30:00 System: Interface ethernet 2, state down
2012-1-1T12:00:00 init: alarm-control (PID 111) terminate signal sent
2012-1-1T12:00:00 init: bslockd (PID 124 ) terminate signal sent
2012-1-1T12:00:00 init: ce-l2tp-service (PID 123 ) terminate signal sent
2012-1-1T12:00:00 init: chassis-control (PID 1111 ) terminate signal sent
2012-1-1T12:00:00 init: class-of-service (PID 11112) terminate signal sent
2012-1-1T12:00:00 init: craft-control (PID 111) terminate signal sent
2012-1-1T12:00:00 init: database-replication (PID 2718932 ) terminate signal sent
2012-1-1T12:00:00 init: diameter-service (PID 2993 ) terminate signal sent
2012-1-1T12:00:00 init: disk-monitoring (PID 7082 ) terminate signal sent?
2012-1-1T00:00:00 %SYS-5-CONFIG I: Configured from console by vty2 (10.11.11.11)
2012-1-1T15:45:10 msg [200] : STP: VLAN 100 Port 22 STP State -> DISABLED (PortDown)
2012-1-1T15:45:10 msg [201] : System: Interface ethernet 22, state down
2012-1-1T16:12:40 System: Interface ethernet 1, state down
2012-1-1T16:12:40 System: Interface ethernet 1, state up
2012-1-1T16:12:40 System: Interface ethernet 2, state down
2012-1-1T20:30:00 init: alarm-control (PID 111) terminate signal sent
2012-1-1T20:30:00 init: bslockd (PID 124 ) terminate signal sent
2012-1-1T20:30:00 init: ce-l2tp-service (PID 123 ) terminate signal sent
2012-1-1T20:30:00 init: chassis-control (PID 1111 ) terminate signal sent
2012-1-1T20:30:00 init: class-of-service (PID 11112) terminate signal sent

: reboot: linkup: linkdown: interface flap

Fig. 3. Image of log grouping.

Log

Root cause of
failurelog grouping, we use the machine learning techniqueknown as non-negative matrix factorization (NMF)
by converting input log data into a matrix (Fig. 3).

Log group

Log templatewith high periodicity
Log templatewith high frequency
Time

Detection
point

Fig. 4. Conceptual image of log visualization.

2.4 Visualization
In this step of the analysis, log data are expressed asa graph. A conceptual image of log visualization isshown in Fig. 4, and an example of a log graph isshown in Fig. 5. In both figures, the horizontal axis
represents time, and the vertical axis represents the
template or log groups mentioned earlier. Each pointin the graph represents the occurrence of each log
template or log group at each time. Hosts are distinguished by their different colors and patterns in thisexample. The order of log templates or log groups on

Fig. 5. Example of log graph (one week’s syslog data).



NTT Technical Review

Feature Articles(number of tweets per 10 min.)
Network-failure related tweets

800
700
600
500
400
300
200
100
0
30 minNetwork failure occurredTime
Restoration

Fig. 6. Time series of tweet counts related to an actual network failure.

the vertical axis is determined according to frequency
and periodicity. That is, the log template with highfrequency is set at the bottom of the vertical axis;
above that is the log template with high periodicity.
The template groups are positioned above them. They
are sorted by log group frequency and then sorted
based on their first appearance. By differentiating log
messages with high frequency or periodicity, we candistinguish the log groups that occur independentlyof time. This makes it possible to visualize millions
of log messages in a single screen and to easily understand in a visual way when the log messages occurred
and what kinds of log messages appeared. Further,sorting by frequency and periodicity makes it easier
to find unusual types of log messages, and grouping
log messages helps associate unstructured log messages with real events that occurred in the network.
3. Twitter analysis
Network operators can monitor network equipment
by using monitoring technology such as SNMP
(simple network management protocol). Although
they can detect hardware failures, it is difficult for
network operators to detect failures caused by software bugs or to detect quality deterioration due tocongestion. Consequently, some cases become silent
failures, which cannot be detected by network operators.
We have studied a way to monitor a social networking service (SNS), namely, Twitter [2], to discoverproblems affecting subscribers. For example, we cansee a surge in tweets about network failures when aVol. 11 No. 11 Nov. 2013

network failure occurs, as shown in Fig. 6. We developed a system to monitor Twitter in real-time bychecking for surges in these kinds of tweets.
3.1 System requirements
Twitter is a popular platform for discussing countless conversation topics, and the number of tweets
now exceeds 400 million per day [3]. Japanese tweets
alone account for 80–100 million tweets per day.
Since the number of tweets that relate to network
problems is very small in the total number of tweets,
we need a way to extract only relevant tweets (firstrequirement). In addition, to detect the area where a
network failure occurs, we need a way to determine
the location of the tweeters (second requirement).
3.2 M
 ethod to extract only network-failure
related tweets
We found in our investigation that keyword matching, a traditional way to search tweets, was not sufficient for automated monitoring because it resulted inmany false positives. This occurs when the tweets
contained the keywords, but the tweets were notrelated to problems with the network. For example, if
we search using the keywords call and drop, we may
get tweets such as: “I dropped my phone in the toilet
so I can’t call or text”. Because keywords such as calland drop are not network-specific words, keyword
matching may lead to a lot of false positive tweetsthat contain the keywords but not the topic of the
network problem.
The network failure detection architecture is shownin Fig. 7. We use supervised learning, namely, SVM
Feature Articles

All tweets

Tweetscontaining
keywords

Keywordmatching

Networkfailure related
tweetsInferredfunction

User name

List of expressions AND List of carriersof network failureand service names

Failure, outage, can’t connect,
slow …

Tweets fortraining data
True class(network failures)

Inferred function by SVM
True class

DisplaysystemLocation
estimation
Estimating a user’s actual locationfrom a tweeted location by kernel
density estimation

Vectorization

False class (notnetwork failures)

False class

Fig. 7. Framework of network-failure detection system using Twitter.

(support vector machine), to suppress the false positives. Supervised learning uses a data set of training
examples. Each training example consists of a pair of
the text of a tweet and a label indicating whether thetweet is related to a network failure. A supervised
learning algorithm analyzes the training data andproduces an inferred function to divide tweets into
those that are related to network failures and thosethat are not. In our approach, each tweet is translated
into a vector by using the bag-of-words method,which is a traditional method in document classification. This method can be expected to suppress thefalse positives by statistically considering all words
appearing in one tweet.
We evaluated the effectiveness of our method byapplying it to an entire year’s worth of Twitter data.
Six network failures were reported by a network carrier in that period. We evaluated the network failure
detection system by counting the number of tweetsthat were classified by our method. When the countexceeded a certain threshold, we considered it to be
an alert of a network failure. We also used the keyword-matching method for comparison. Both methods detected the 6 actual network failures. However,
the keyword-only method also falsely detected 94
events, whereas the machine-learning method suppressed almost all of those and had only 6 false detections.
3.3 Method to determine the location of tweeters
Twitter has a function to attach the user’s location
by GPS (Global Positioning System) data, but mostusers choose not to opt into this function. Therefore,
we need to estimate the location of Twitter users who
wrote the network-failure-related tweets. Some studies have used the bias of a distribution of words,
which mainly involves dialect characteristics, to estimate a user’s location.
However, these studies estimate a rough granularity
of areas such as the Kanto region with an error of
about 150 km and do not meet our requirement,
which is to achieve at least prefecture-level location(an error of less than 50 km).
Therefore, we studied a high-accuracy location
estimation method that uses gazetteer information,
which includes the pairs of a geographic name and its
coordinates. While most tweets do not contain GPS
information, many tweets contain a geographic name.
Although users may tweet the geographic names ofplaces other than where they are actually located, theoverlapped locations of many of their tweets will
make it possible to estimate their location because
Twitter is a service for users to post what they are
doing. We used the kernel density estimation method
to overlap the tweets of individual tweeters. We
evaluated the estimation error of users whose locations were known and found that the estimation error
was less than 50 km for two-thirds of those users.
Furthermore, the estimation error was less than 25 km
for half of all users, which demonstrated that our
method was effective.

NTT Technical Review

Feature Articles

4. Conclusion
We introduced a big-data approach consisting ofsyslog and SNS analysis to predict or detect network
failures. In cooperation with group companies, we arenow evaluating the efficiency of syslog analysis using
actual syslog data. We are also preparing a proposal
for group companies for the use of SNS analysis as atool for detecting silent failures.

References
[1]
[2]
[3]

D. D. Lee and H. S. Seung, “Learning the Parts of Objects by NonNegative Matrix Factorization,” Nature, Vol. 401, No. 6755, pp.
788–791, 1999.
Twitter.com.
https://twitter.com/?lang=en
Celebrating #Twitter7.
http://blog.twitter.com/2013/03/celebrating-twitter7.html

Tatsuaki Kimura

Masahiro Yokota

Kei Takeshita

Ken Nishimatsu

Researcher, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the B.E. degree in informatics andmathematical science and the M.I. degree insystem science from Kyoto University in 2006
and 2010, respectively. Since joining NTT in
2010, he has been engaged in research on traffic
measurement and management of large-scale
networks. He is a member of the Operations
Research Society of Japan (ORSJ) and the Institute of Electronics, Information and Communication Engineers (IEICE).

Communication Traffic & Service Quality
Project, NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in engineering from Keio University, Kanagawa, in2009 and 2011, respectively. Since joining NTT
in 2011, he has been engaged in researching traffic management. He is a member of IEICE.

Researcher, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the M.E. degree in information
science from Osaka University in 2008. He
joined NTT Service Integration Laboratories(now NTT Network Technology Laboratories) in2008 and studied network design of IP networks.
He is currently studying the increasing complexity of network operations by data analysis. He
received the 2012 IEICE Young Researcher’sAward. He is a member of IEICE.

Senior Research Engineer, Supervisor, Communication Traffic & Service Quality Project,
NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in engineering from Waseda University, Tokyo, in 1995
and 1997, respectively. Since joining NTT in
1997, he has been engaged in telecommunication
traffic analysis and modeling customer-servicechoice behavior. He is a member of IEICE andIPSJ.

Tsuyoshi Toyono

Tatsuya Mori

Research Engineer, Communication Traffic &Service Quality Project, NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in information science from Keio University, Kanagawa,in 2001 and 2003, respectively. He joined NTT
Information Sharing Platform Laboratories (nowNTT Network Technology Laboratories) in2003. He has been researching IPv6 network
technology and distributed information management systems. His current research interests
include the operation and management of largescale networks. He is a member of the Information Processing Society of Japan (IPSJ).

Vol. 11 No. 11 Nov. 2013

Associate Professor, Waseda University.
He received the B.E. and M.E. degrees inapplied physics, and the Ph.D. degree in information science from Waseda University, Tokyo, in
1997, 1999, and 2005, respectively. Since joining
NTT in 1999, he has been engaged in researchingmeasurement and analysis of networked systems
and network security. From Mar. 2007 to Mar.
2008, he was a visiting researcher at the University of Wisconsin-Madison, Madison, WI, USA.
He received the Telecom System Technology
Award from Telecommunications Advanced
Foundation in 2010 and the Best Paper Awards
from IEICE and IEEE/ACM COMSNETS in2009 and 2010, respectively. He is a member of
IEICE, the Association for Computing Machinery, and IEEE.

Feature Articles: Applications of Big Data Analytics
Technologies for Traffic and Network Management Data

Analyzing Macroscopic Demand
Structure Based on New Model ofS-shaped Growth
Shinsuke Shimogawa, Kei Takeshita, and Ken Nishimatsu
Abstract

We tackle the issue of quantitatively evaluating the impact competitors have on a market. In this article, we introduce a method for estimating this impact based on a new theory of S-shaped growth ininnovation diffusion.
Keywords: S-shaped growth, structural change, innovation diffusion

1. Introduction
It is important for network planners to forecast thedemand for network traffic at high resolution. Whereas short-term forecasting, for example, within a oneyear range, may be possible by extrapolating it linearly from past demands, medium- and long-term
forecasting is much more difficult.
The growth curves of the number of people using a
new service are often S-shaped. That is, growth issmall when the service is first introduced; it increases
soon after at an accelerated pace, and then slows
down in the final stage as the service becomes morewidespread. The S-shaped growth curve is often
observed with new services, products, methods, orideas. However, the structure of the S curve has notbeen revealed despite the large number of studies thathave been done on it.
Studies on the diffusion of innovations, a conceptthat involves how, why, and at what rate new ideas
and technology spread through cultures, began in the
19th century. Empirical studies [1] have revealed
important facts including that the population ratio of
earlier adopters of an innovation is small and that theytend to communicate about the innovation with people beyond their local communities, whereas later
adopters tend to communicate about the innovation
inside their local communities. However, even thebasic question of why growth curves are often Sshaped had not been answered.
Most of the proposed models to explain or capture
the S shapes were given by mathematically explicit
formulas with few parameters. Many studies examined these models with diffusion data and showed
fairly small errors of fitting with the data. However, it
has been revealed that these models have rarely succeeded when they are applied to forecasting the
growth over a long term. We show how such forecasting fails by extrapolating a traditional model, namely
the Bass model [2], in Fig. 1. The graph in this figure
shows the subscribers of a fixed broadband accessservice. The blue lines are forecasting results obtainedby extrapolation using the Bass model with different
times of starting the forecasting. The results vary
greatly when the time to start the extrapolation ischanged, and all of the results of long-term forecasting eventually fail. In contrast, the results achieved
using our method (red line) fit the actual numbers
quite well.
NTT Network Technology Laboratories has been
investigating this issue, and we recently proposed a
new theory [3] that explains the S-shaped growth.
Our new theory offers at least two findings regarding
forecasting. First, the final adopter population cannot
be estimated by extrapolation of growth curves. Second, stable long-term forecasting is possible until thedemand for something reaches the final stage. In thisarticle, we give an overview of our new theory and
NTT Technical Review

Feature Articles

Our theory
α log t, t > inflection point

3.5E+07
Subscribers of fixed broadband
access service (–2008)
Dec. ’06
Dec. ’05
Dec. ’04

2.5E+072.0E+07m

Dec. ’03

1.5E+07
1.0E+07m

Data−(p+q)t1−e−(p+q)tq
1+ p e5.0E+06

Long-term forecasting is difficult
with traditional method.

’09

’08

’06

’05

’04

’03

’02

’01

(Bass)’00

0.0E+00’07

No. of subscribers

3.0E+07

Year

Fig. 1. Failure of forecasting using traditional method.

describe how it can be applied to analyze the demandfor broadband service in the Japanese market.
2. New theory of S-shaped growth
The key to our new theory follows the basic hypothesis: the heterogeneity of adopters follows powerlaws. The hypothesis regarding the heterogeneity of
adopters has been proposed in previous studies [1].
Although the researchers in those studies assumed
mathematically tractable distributions of heterogeneity such as normal distributions, our hypothesis
adopts a pair of power distributions. This pair of
power distributions has been widely observed; e.g.,
the distribution of income and that of the frequencies
of using content. These distributions have discontinuities that consist of a boundary point between thedomains of two power distributions and the upper andlower cut-offs of the distributions. These two power
distributions indicate that a society can be seen as
consisting of a strong minority and a normal majority
when we evaluate individual performance using acertain measure.
We can derive a structure of S growth on three levels: the structure of individual adoptions of an innovation, the social structure of the flows of information
about the innovation, and the S shape of a growth
curve. The social regularities (power laws) are represented as the characteristics of the flows of innovation information. Our new theory clarifies the macromicro relationship between the structure of individual
adoptions of innovations and the S shape of a growth
curve, which will provide a new basis for understanding the diffusion of an innovation.
Vol. 11 No. 11 Nov. 2013

We can describe the net increase in the demand fora certain service by using two different power-law
distributions, which transit at an inflection point on
the S growth curve, as shown in Fig. 2. An inflection
point is a point that represents the maximum value ofa net increase.
The forecasting model for the days after the inflection point can be formulated as y(t) = alog(t)+y0,
where t represents the days that have elapsed since
the first day of an innovation diffusion, y(t) is the total
demand at t, a and y0 are constants, and the base of
the logarithm is natural. This formula stably extrapolates the number of subscribers of broadband andmobile phone services.
3. Analysis of demand for a broadband service
Our new theory enables us to forecast the numberof subscribers of a broadband service if the market
structure does not change. Conversely, a large error
between the forecast and the actual values means thata change in the market structure occurred. The number of subscribers of a fixed broadband service isplotted in Fig. 3 as a red dotted line. Since the growth
curve can be described by y(t) = alog(t)+y0 after the
inflection point, the number of subscribers grows
linearly on the logarithmic x-axis. The number ofsubscribers grew along the line, and then the red line
changed its gradient twice, which means market
changes occurred. We infer that these changes in themarket situation were due to mobile broadband services that started diffusing at those times.
We can quantitatively analyze the change in themarket as seen in the graph, which will enable us toFeature Articles

7.0
α log t3.0E+072.5E+07

R2 = 0.9996

2.0E+07
Inflection
point

1.5E+07

6.8

No. of subscribers (logarithmic)

No. of subscribers (natural)

3.5E+07

1.0E+07
5.0E+06

2.1

2.3

2.5

2.7

2.9

3.1

3.3

1.7–2

6.4

tβ

6.2
6.0
5.8

0.0E+00
1.9

β

6.6

5.6

y = 1.695x + 1.9879

5.4

R2 = 0.9973

5.2

3.5

1.8

Elapsed days (log)

2

2.2

2.4

2.6

2.8

3

Elapsed days (log)log: logarithmic scaleFig. 2. F orecasting model applied by using discontinuity and two different power laws.

No. of subscribers, in units of 10,000

5000
Growth curve of broadband
services in Japan

4500
4000
3500

DSL + FTTH + CATV
+ BWA + E-mobile

3000

DSL + FTTH + CATV+ BWA
DSL + FTTH + CATV

2500
2000

Mar. ’03

1000
500
0

Mar. ’08 Sept. ’10

Mar. ’07
Mar. ’06
Mar. ’05

Mar. ’04

1500

Sept. ’11
Sept. ’10
Mar. ’09

Mar. ’08

Sept. ’11

Mar. ’02
Mar. ’01
6

6.5

Mar. ’08
7

7.5

BWA: broadband wireless access
CATV: community access television

Mar. ’09

8

Elapsed days (log)Sept. ’10

8

DSL: digital subscriber line
FTTH: fiber to the home

Fig. 3. Impact of mobile broadband services to fixed broadband services.

develop an effective strategy and to allocate costs
appropriately. We can identify the change by inspecting the red dotted line showing the number of subscribers on a logarithmic axis (Fig. 4). We found thatwhile the curve representing a nationwide scale
described only high-impact changes, the curve representing the prefecture scale showed differences in thecompetition circumstances of each prefecture.



4. Conclusion and future work
We expect our method to be utilized for planning
network and service strategies effectively and in atimely manner considering that the competition in theJapanese broadband market is intensifying for both
fixed and mobile services. Our goal is to model themarket behavior in order to predict the efficiency of
supply-side actions. As the first step, we are planning
NTT Technical Review

Feature Articles

of service Aof service B

Actual – Forecast

Number ofsubscribers
3.5E+073.0E+072.5E+072.0E+07
1.5E+07

α Log tR = 0.9996=

Impactof service C+

Impact+Impactof service E+

ImpactReduction due
to service C

Forecasting lineof service A

ServiceA

Reduction due toservice B

Impact of service B

2of service D

Which competitor affected our service?

Inflection
point

1.0E+07
5.0E+06

Linealized by atheoretical model
0.0E+00
1.9 2.1 2.3 2.5 2.7 2.9 3.1 3.3 3.5
Elapsed days (log)

Forecasting line of service A
After introduction of service B and C
Elapsed days (log scale)Service
B

Service C

ServiceD

ServiceE

Fig. 4. Analysis of factors affecting broadband service.

to analyze the market on a more granular level tolearn more about the behavior of individual consumers.

References
[1]
[2]
[3]

Shinsuke Shimogawa

Manager, Network Technology Center, NTT
Advanced Technology Corporation.
He received the B.Sc. and M.Sc. degrees inmathematics from Osaka University and the
Ph.D. degree from Kyushu University, Fukuoka,
in 1984, 1986, and 1994, respectively. He joined
NTT in 1986. His research interests include traffic, networks, platforms, individual and organizational behaviors, design strategy, performance
prediction of enterprise actions, socio-dynamics,
and general structural changes that occur with
innovations. He is currently trying to understandsuch changes from a universal hypothesis interms of dissipative structure and complex
dynamics.

E. M. Rogers, “Diffusion of Innovations,” 1st ed. Free Press, New
York, 1962.
F. M. Bass, “A New Product Growth Model for Consumer Durables,”Manage. Sci., Vol. 15, No. 5, 1969.
S. Shimogawa, M. Shinno, and H. Saito, “Structure of S-shaped
growth in innovation diffusion,” Phys. Rev. E, Vol. 85, 056121,
2012.

Ken Nishimatsu

Senior Research Engineer, Supervisor, Communication Traffic & Service Quality Project,
NTT Network Technology Laboratories.
He received the B.E. and M.E. degrees in engineering from Waseda University, Tokyo, in 1995
and 1997, respectively. Since joining NTT in
1997, he has been engaged in telecommunication
traffic analysis and modeling customer-servicechoice behavior. He is a member of IEICE andthe Information Processing Society of Japan.

Kei Takeshita

Researcher, Communication Traffic & Service
Quality Project, NTT Network Technology Laboratories.
He received the M.E. degree in information
science from Osaka University in 2008. He
joined NTT Service Integration Laboratories(now NTT Network Technology Laboratories) in2008 and studied network design of IP networks.
He is currently studying the increasing complexity of network operations by data analysis. He
received the 2012 Institute of Electronics, Information and Communication Engineers (IEICE)
Young Researcher’s Award. He is a member of
IEICE.

Vol. 11 No. 11 Nov. 2013



Regular Articles

Fiber-mounted Electro-optic Probefor Microwave Electric-field
Measurement in Plasma Environment
Hiroyoshi Togo
Abstract

The demand for greater precision in electric-field measurements is increasing. To meet this demand,
NTT Microsystem Integration Laboratories has developed an electro-optic (EO) probe that is mounted
on the tip of an optical fiber and has no metal components. This article describes the basic principles andimprovements in sensitivity stability and thermal and magnetic resistance of the EO probe. It alsopresents the results of microwave electric-field measurements conducted in an electron cyclotron
resonance ion engine installed on an asteroid explorer under the collaboration of NTT and the Japan
Aerospace Exploration Agency.
Keywords: electro-optic (EO) probe, optical fiber, plasma environment

1. Introduction
Electric-field sensors are widely used to measurevarious kinds of electromagnetic fields such as weakones existing naturally in the environment and artificial ones generated by high-power microwaves.
Measurement of these fields usually disturbs the radiated electromagnetic waves. Conventional electricfield sensors contain metal components that canincrease these disturbances, and are approximately aslarge as a wavelength that can determine a spatial
resolution which is an inherent problem whenattempting to precisely measure the desired electricfield. To overcome this issue, at NTT Microsystem
Integration Laboratories, we have developed an electro-optic (EO) probe that employs an optical fiber
typically used in optical communications [1], [2]. The
EO probe was designed based on the Pockels effectand contains no metal; thus, it can reduce the disturbances in the electromagnetic field that occur during
measurement. Moreover, the size of the EO probe issmall and can be determined based on the opticalbeam propagating in an EO crystal.
Moreover, the Japan Aerospace Exploration Agency (JAXA) has been researching and developing anelectron cyclotron resonance (ECR) ion engine thatcan be installed in an asteroid explorer and has also
improved its propulsion performance [3], [4]. To
improve the performance, JAXA needed to be able tomeasure the microwave electric-field distribution of
the internal plasma in the ion engine during actual
operation. However, conventional electric-field sensors cause disturbances such as scattering of the electric field, which makes it necessary to modify the
structure of the ion engine in order to insert a probe in
it. This makes it difficult to precisely diagnose internal phenomena based on microwave electric-fieldmeasurements of the accelerated plasma in the ion
engine during operation.
Thus, NTT Microsystem Integration Laboratories
and JAXA have been collaborating in the researchand development of microwave electric-field measurements in the ion engine with a focus on achieving
noninvasiveness and high spatial resolution of the EO
probe. The configurations and basic properties of the
EO probe are described in section 2, and the improvements made to the probe are explained in section 3.
The experimental setup is described in section 4, andin section 5, the results of the microwave electricfield measurements in the ion engine during actual
NTT Technical Review

Regular Articles

EO crystal

Mirror
1 mm

Lens

FR

Optical source

FerruleOptical polarization
controller and detector

Optical fiber(10 m)

Fig. 1. Fiber-mounted EO probe system.

operation are presented.
2. Fiber-mounted EO probe
A photograph of the EO probe developed by NTT
Microsystem Integration Laboratories for the microwave electric-field measurements is shown in Fig. 1.
The EO probe employs photonic techniques based on
the Pockels effect to detect the microwave electricfield. It consists of a dielectric reflector, an EO crystal, a Faraday rotator (FR), a collimator lens, a ferrule, and a polarization-maintaining fiber (PMF). The
reflector, EO crystal, FR, lens, and ferrule are longitudinally mounted on the tip of the PMF with opticaladhesive that has a refractive index as high as that offused quartz. The diameters of these mounted components are 1 mm, while that of the PMF is 0.256 mm,
and the lengths of the EO probe and the PMF are
15 mm and 10 m, respectively. The EO crystal is zinctelluride (ZnTe) with a Pockels coefficient of around
4 pm/V at an optical wavelength of 1550 nm. A lightwave propagates through these components without
any distortion at zero electric field. In this probe, therefractive index along the crystallographic axes of theEO crystal changes, and then the phase of the lightwave propagating along each axis in the EO crystal
shifts in linear proportion to the intensity of theapplied microwave electric field. That is, the refractive index of the EO crystal depends on the opticalpolarization relative to the crystal axes. ZnTe has a
cubic zinc-blende lattice, and the polarization of thelightwave propagating normal to the (110) planechanges as the microwave electric field is applied tothe axis normal to the (001) plane. The microwave
electric field can be measured by detecting the polarization change of the lightwave propagating roundVol. 11 No. 11 Nov. 2013

Mass flow
controller
Propellant
tank

Oscillator

Grid system

Waveguide

Discharge
chamber

Antenna

MagnetMicrowave
amplifier
CathodeFig. 2. ECR Ion Engine.

trip through the crystal in the EO probe. In Fig. 1, themeasurement direction of the microwave electricfield is transversal to the longitudinal one of the EO
probe.
3. ECR ion engine installed in asteroid explorer
A schematic diagram of the ECR ion engine thatwas installed in the asteroid explorer developed by
JAXA is shown in Fig. 2. The ECR ion engine basically consists of a waveguide, a discharge chamber,and a three-grid system. Inside the ECR ion engine,
there is a dipole antenna and a propellant inlet at one
end of the waveguide. The other end of the waveguideis connected to the discharge chamber (right side in
Fig. 2). The ECR ion engine uses a microwave with afrequency of 4.25 GHz to produce plasma, which istransmitted through the waveguide to the dischargeRegular Articles

Improvements

Issues in microwave plasma measurement:(1) Improvement of sensitivity stability

16%

10%

(2) Improvement of magnetic field resistivity

0.1 T

0.8 T

(3) Improvement of thermal resistivity

100ºC

1200ºC(1) Low-coherence optical source was used to suppress opticalresonance phenomena in the optical fiber.

Reflection ~ 0.2%

Reflection ~ 0.1%

Glass tube

Optical fiber

Polarization
controller

Detector

EO crystal(2) Optical components weak againsthigh magnetic fields were redesigned.

Water
coolerOpticalsource(3) Water cooler anddouble capillary
structures were used.

Fig. 3. Improvements of EO probe.

chamber. The 4.25-GHz microwave and a propellant,
xenon (Xe), are injected from the waveguide andtravel to the discharge chamber. In the dischargechamber, Xe electrons are continuously accelerated
by the ECR heating of the microwave and the mirror
magnetic field generated by two rings of magnets.
ECR plasma is formed through subsequent electronneutral and electron-ion collisions. JAXA has
improved the thrust force of the ECR ion engine bychanging the way the propellant is injected [3], [4]. In
conventional ECR ion engines, the propellant inlet is
located only at the end of the waveguide. In the
improved ECR ion engine, propellant inlets are added
between the magnet rings in the discharge chamber.
4. Improvements to EO probe formeasurements in ECR ion engine
As described in the previous section, the environment inside the ECR ion engine is very severe, andthus, the EO probe needed to be modified to improve
its performance in carrying out the specified measurements in the ECR ion engine during actual operation, as shown in Fig. 3.
(1)	To achieve stable measurement with less fluctuation than that of a microwave oscillator output, a low coherence optical source was used tosuppress the optical resonance phenomena in
the optical fiber. Accordingly, the sensitivity


fluctuations of the fiber-optic EO probe werereduced from 16% to less than 10%.
(2)	To enable the EO probe to be applied to highmagnetic-field areas in the ECR ion engine, theoptical components that were weak against highmagnetic fields were redesigned. This increased
the magnetic-field resistance from 0.1 T to over
0.8 T.
(3)	To enable application of the EO probe to hightemperature areas, a water cooling system witha double capillary structure was introduced [5].
The outer diameter of the capillary structure is
3 mm, which made it possible to insert it into the
ECR ion engine through the grids without
requiring any additional perforations or modifications.
5. Measurement setup
The experimental setup [6] used to conduct theelectric-field measurements inside the ECR ion
engine is shown in Fig. 4. In the vacuum chamber, theEO probe is placed in a silica glass tube 3 mm indiameter, which is mounted on a linear stage and arotary stage positioned by stepping motors. The
probe is remotely inserted in the discharge chamber
through the center aperture of the grids. The measurement interval is 1 cm. The rotational angle is adjusted
so that the measurement direction of the EO probe is
NTT Technical Review

Regular Articles

parallel to the longitudinal direction of the dipole
antenna, where the maximum electric field is
detected. The input microwave power is 34 W.
A laser diode (LD), polarization controller, photodetector (PD), and spectrum analyzer are set outside
the vacuum chamber. The polarization controller
comprises a polarizer, a circulator, half and quarterwave plates, and an analyzer. The lightwave emitted
from the LD is polarized linearly by the polarizer andinput into the PMF; it reaches the tip of the probe
through the circulator. The lightwave makes a roundtrip pass through the EO crystal via reflection by thedielectric reflector, and the polarization of the lightwave changes in proportion to the microwave electricfield along the measurement direction. The polarization-changed light is returned to the polarization

Vacuum chamber
Rotarystage
EO probe

controller and converted to linearly polarized light by
the analyzer. That is, the lightwave is intensity-modulated by the microwave electric field. The intensitymodulated light is detected and converted to an electric signal by the PD. The amplitude and phase of theelectric signal are proportional to those of the microwave electric field, and the frequency of the electric
signal is the same as that of the microwave electricfield. As a microwave signal is input into the ion
thruster at a frequency of 4.25 GHz, the spectrum
analyzer displays the spectrum with the intensity proportional to that of the measured microwave electricfield at a frequency of 4.25 GHz.
Prior to the measurement, the probe is calibratedusing a rectangular waveguide with a cross section of
2.0 × 4.0 cm2 in atmospheric pressure. The calibration results agreed well with the theoretical electricfield intensity in the waveguide at a known microwave power.
In the measurement, the EO probe was kept at 25°C
by a cooling system under beam acceleration in orderto achieve the same accuracy as the calibration. The
measurement was conducted at various points inside
the thruster, as shown in Fig. 5.
6. Electric-field measurement [7]Linear stage
Laser
Photodetector

Polarization
controller

The intensity distribution of a microwave electricfield parallel to the dipole antenna in the ECR ion
engine is shown in Fig. 6. The measurement was conducted with respect to the propellant inlet positions of
the waveguide and the discharge chamber at a flowrate of 2 sccm under beam acceleration. Each graphhas a different scale of intensity to express the differences in the intensity of the electric field between thecenter axis and the ECR region.

Polarization
maintaining fiber

Spectrum analyzer

Fig. 4. Measurement setup.

Distance from screen grid, cm
−15

−10

−5

0

1.5

Antenna and measurement direction

2.5
4.5

Radial position, cm

zrFig. 5. Measurement points.

Vol. 11 No. 11 Nov. 2013

Gas injection

Grid facet
0
0
15
25

Microwave power does
not reach yoke becauseof microwave cutoff
phenomena.

45

From discharge chamber

0
15
25

Improvement mechanism
Microwave power increased in waveguide
propulsive power improved by 20%

Gas
injection

Waveguide

45

1

0.5

0

0

Relative intensity(arb. units)

Horizontal position (mm)100
50Relative intensity(arb. units)

150

Radial position (mm)From waveguide

Radial position (mm)Regular Articles

Discharge chamber

Fig. 6. Electric-field distribution inside ion engine (lower half of engine).

As shown in the figure, the intensity distribution of
the electric field with plasma is completely different
between the propellant inlet positions. From thewaveguide inlet, the highest value was recorded at
100 mm from a grid facet. By contrast, from the discharge chamber inlets, the highest value is locatedmore downstream. Moreover, the absolute value ofthe propellant injection from the discharge chamber
inlets is half of that from the waveguide inlet. Thisphenomenon resulted in the saturation of beam current at 2 sccm from the waveguide inlet. The highest
intensities exist at over 12,000 V/m around 100 mm
from the grid facet. At that point, the intensities of the
ECR region are quite low. Hence, it is highly possiblethat insufficient microwave power is supplied to the
ECR region, which results in the reduction of thebeam current.
This measurement revealed that the difference inthe low-beam current mode and the high-beam current mode corresponds to the presence of the peak of
the electric-field intensity of the microwave in the
waveguide. This peak is possibly suppressed bychanging the way the propellant is injected from thewaveguide inlet to the discharge chamber inlets. This
result indicates that the tendency is the same as thatin the laser absorption spectroscopy of Xe I 828.01
nm [4], which can explain the mechanism of improvement of the propellant force in the ECR ion engine.

cal components in the probe because of their complicated design structure and the long-term reliability ofthe optical adhesive resin used to fix the optical components such as the EO crystal and the optical fiber.
NTT Microsystem Integration Laboratories aims to
improve the operability by using a novel structure forthe fiber-mounted EO probe and to increase theprobe’s reliability by improving the optical adhesive
resin. The last step will be to finalize the design of thefiber-mounted EO probe used for obtaining microwave electric-field measurements within the microwave discharge ion engine.
References
[1]

[2]

[3]

[4]

[5]

7. Future tasks
We are now considering the operability of the opti

[6]

H. Togo, N. Kukutsu, N. Shimizu, and T. Nagatsuma, “SensitivityStabilized Fiber-Mounted Electrooptic Probe for Electric Field Mapping,” Journal of Lightwave Technology, Vol. 26, No. 15, pp. 2700–
2705, 2008.
H. Togo, S. Mochizuki, and N. Kukutsu, “Optical Fiber Electric Field
Sensor for Antenna Measurement,” NTT Technical Review, Vol. 7,
No. 3, 2009.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2009
03sf5.html
K. Nishiyama, T. Nakai, and H. Kuninaka, “Performance Improvement of the Microwave Discharge Ion Thruster u20,” Proc. of the
42nd AIAA/ASME/SAE/ASEE Joint Propulsion Conference &Exhibit, AIAA 2006-5176, Sacramento, California, USA, 2006.
R. Tsukizaki, H. Koizumi, S. Hosoda, K. Nishiyama, and H. Kuninaka, “Improvement of the Thrust Force of the ECR Ion Thruster
μ10,” Trans. JSASS Space Tech. Japan, Vol. 8, pp. Pb 67–Pb 72,
2010.
T. Ise, R. Tsukizaki, H. Togo, H. Koizumi, and H. Kuninaka, “Electricfield measurement in microwave discharge ion thruster with electrooptic probe,” Review of Scientific Instruments, Vol. 83, No. 12,124702-1-6, 2012.
T. Ise, R. Tsukizaki, H. Togo, H. Koizumi, and H. Kuninaka,

NTT Technical Review

Regular Articles[7]“Elucidation of Internal Phenomena in Microwave Ion Thruster μ10
Utilizing Electro-Optic Fiber Probe,” Proc. of Space Transportation
Symposium, STEP-2012-040, Kanagawa, January 2013 (in Japanese).
R. Tsukizaki, T. Ise, H. Koizumi, H. Togo, K. Nishiyama, and H.
Kuninaka, “Plasma Diagnostics of the μ10 ECR Ion Thruster Using
Optical Fiber Probes,” Proc. of 33rd International Electric Propulsion
Conference (IEPC2013), IEPC-2013-270, Washington D.C., USA.

Hiroyoshi Togo

Senior Research Engineer, Smart Devices
Laboratory, NTT Microsystem Integration Laboratories.
He received the M.Sc. and Ph.D. degrees inapplied physics and electronic engineering fromthe University of Tsukuba, Ibaraki, in 1996 and2010. He joined NTT Musashino Opto-electronics Laboratories in 1996. From 1996 to 2001, he
was engaged in the development of a thermocapillary waveguide-based optical switch, andfrom 2001 to 2002, he worked on commercializing it for NTT Electronics Inc. Since 2002, he has
been researching ultrawideband impulse radio
systems using photonic techniques and millimeter-wave tomography with electro-optic probing.
He is a senior member of the IEEE Photonics,
Antenna and Propagation, and Microwave Theory and Techniques Society, and the Institute ofElectronics, Information and Communication
Engineers (IEICE). He served as the financedirector of the Council of Technical Committee
Representatives in the IEICE Communications
Society from 2011 to 2012, and has been an
expert member of the Technical Committee on
Microwave Photonics in the IEICE Electronics
Society since 2005 and the Committee on Photonics-applied Electromagnetic Field Measurements in the IEICE Communications Society
since 2012. He received the 2006 Asia PacificMicrowave Photonics Conference AP-MWP
Award and the 2010 European Conference on
Antenna and Propagation Award.

Vol. 11 No. 11 Nov. 2013



Global Standardization Activities

International Standardization Trends
in China
Daisuke Ikegami
Abstract

In China, independent technological innovations are progressing in accordance with the National
Guideline for Medium and Long-term Plan for Science and Technology Development, announced by theState Council of the People’s Republic of China in 2006. In the context of such innovation, standardization activities are under way in various industrial fields, with specific goals set for each area of technology. Standardization in the communications industry is being implemented by the China Communication Standards Association (CCSA). This article reviews recent developments in the CCSA’s standardization activities and explains the structure of the Chinese standardization system and recent trends inthe communications industry.
Keywords: standardization trend, CCSA, M2M

1. Introduction
Chinese telecommunications technology has maderemarkable strides in recent years. Information andcommunications technology (ICT) is rapidly becoming widespread, with the number of mobile phone
subscribers and broadband subscribers respectively
reaching about 1.1 billion and 175 million as of theend of 2012. Standards have played an important rolein the diffusion of broadband. The national compulsory standard GB*1 50846-2012 Design Specification
for FTTH (fiber to the home) Facilities in Residential
and Construction Areas, enacted this year on April 1,
deserves particular attention. The specification makes
FTTH wiring compulsory for each household in new
multi-unit apartments and residential buildings in cities where FTTH is available. The enforcement of thiscompulsory standard is expected to make FTTH evenmore widespread.
In addition to the promotion of broadband, systematic measures are being devised in a variety of fields
following the uniquely Chinese model in which
political instruction is given by the government on
which direction to take. In China, the central government lays out its policies every five years and determines which areas to focus industrial development
on. The period from 2011 to 2015 is covered by the
12th Five-Year Guideline, which is focusing on sevenindustrial fields as key areas: energy conservation/
environmental protection, next-generation information technology, biotechnology, the production of
cutting-edge equipment, new energies, new materials, and cars employing new energy sources. Macrodevelopment plans to develop the industrial fields
established in the Five-Year Guidelines are also subsequently announced with the release of each set of
guidelines. In response to these development plans,
ministries, agencies, and local governments also set
numerous guidelines and plans that include numerical targets and conceptual levels to be achieved by theend of the 12th Five-Year Guideline period (that is, by
the end of 2015) [1].
Having joined the WTO (World Trade Organization) in 2001, China is working to ensure transparency in its compulsory standards, optional standards,
and conformity evaluation procedures in accordancewith the TBT (Technical Barriers to Trade) agreement imposed on member countries. At the sametime, it will need to establish domestic regulations by
incorporating international standards and guidelines.
*1	 GB stands for Guobiao, Chinese for national standard.

NTT Technical Review

Global Standardization Activities

2. National Guideline for Medium andLong-term Plan for Science andTechnology Development
In February 2006, China officially announced its
National Guideline for Medium and Long-term Planfor Science and Technology Development––a mid- tolong-term plan for scientific and technologicalfields––as a national strategy to cover the period until
2020, and has steadily been implementing it. Thisplan’s four pillars are independent innovation, leaps
in key areas, support for development, and drive
toward the future. China will promote original innovation (leveraging China’s own capacity for innovation), collective innovation (that is, capacity forinnovation by leveraging resources it holds intellectual property rights to), and re-innovation (that is,
capacity for innovation brought about by the introduction of new technologies) and basic technologicalresearch. In doing so, China is aiming to bring about
new breakthroughs, support sustainable social development, and aid the diffusion of basic research andcutting-edge technologies.
In this context, specific key areas and high-priority
tasks have been established and are steadily being
implemented.
3. 12th Five-Year Guideline for Standardization
Since the announcement of the 12th Five-Year
Guideline for Standardization (December 2011) inthe standardization field, a wide array of standardization activities have been underway in China in linewith its long-term targets. Examples of the key areasof standardization in the next-generation information
technology industry are listed in Table 1.
China is working on international standardization
with unprecedented energy, and the number of posts
assumed by Chinese industry leaders in organizations
such as ISO (International Organization for Standardization), IEC (International Electrotechnical Commission) and ITU (International Telecommunication
Union) has been rising year after year.
The number of patent applications has also been
rapidly increasing in a variety of fields, with the number of international applications filed in 2012 under
the PCT (Patent Cooperation Treaty) reaching 18,627(up 13.6% over 2011), placing China in fourth position behind the U.S., Japan, and Germany [2]. China
has also made its presence felt in the field of technological development. Representative examples ofstandards recently proposed by China to international
Vol. 11 No. 11 Nov. 2013

standardization institutions include TD-SCDMA(Time Division-Synchronous Code Division Multiple
Access), IGRS (Intelligent Grouping and Resource
Sharing), WAPI (Wireless-LAN (local area network)
Authentication and Privacy Infrastructure) and TDLTE (Time-Division Long-Term Evolution).
4. Standardization in China
Standardization in China is prescribed by law in theStandardization Law of the People’s Republic ofChina (PRC) (enacted in 1989). All standards currently in effect were established based on this law.
These regulations classify Chinese standards into
four types: national standards, departmental standards (industry standards), local standards, and corporate standards [1]. Each type of standard is further
divided into compulsory standards and optional standards (recommended standards). Standardization
documents for compulsory national standards are
identified by a code number preceded by GB, whereas those for optional standards are indicated by GB/T.
Thus, the status of the standards that are establishedcan be understood by looking at their codes [1].
National standards are established and managed by
SAC (Standardization Administration of China), an
external bureau of the AQSIQ (General Administration of Quality Supervision, Inspection and Quarantine of PRC), while industry standards are enacted
and managed by the central government’s relevant
departments for each industry. Communicationsrelated standardization is carried out under the lead of
MIIT (Ministry of Industry and Information Technology). Regional standards are those established byindividual regions (provinces, autonomous regions,and directly controlled municipalities), and are managed by the standards office of each region’s Bureauof Quality and Technical Supervision.
The organizations that oversee the various standards are shown in Fig. 1. Note that SAC is a member
institution of ISO/IEC.
5. China Communications Standards
Association (CCSA)
5.1 Overview
CCSA was founded in 2002 as the only standardization institution in charge of standards in the Chinese communications industry [3]. CCSA members
come from diverse fields in the communicationsindustry, for example, research institutions, communications carriers, vendors, and universities. TheGlobal Standardization Activities

Table 1. Key points for standardization of next-generation IT industry.
Item

Descriptions

Information network
infrastructure andnext-generation mobile
communication- Establish standards concerning the optical transmission of high-speed multi-services.
- Establish standards for the reinforcement of 2G and 3G mobile communication technology, 4G mobile
communication, CDMA communication, digital honeycomb communications, network security, optical fiber, theunion of fixed and mobile networks, wireless LAN, wireless broadband access, short-distance communication, etc.

Next-generation
Internet coreequipment and smart
terminals- Establish standards for improved network equipment efficiency and security, IPv6, domain systems and security,
software switch security, Internet data centers, individual information processing equipment, e-books, etc.

Triple play

Establish standards for home networks, audio codecs, video codecs, basic standards and contents security
shared by bidirectional digital television platforms and integrated systems, content formats for streaming media,
network television, multimedia telephones, standards of interactive services among third screens (television,
personal computers and mobile phones), IP phones, smart television connection standards and networktransmission security standards for triple play

Integrated circuits andnew-type displays

Establish standards for smart card web servers, electronic ID systems operable with smart cards, general IC cardand smart card security and system security; electricity semiconductor, MEMS product and process standards;
standards for 3D display comfort, compatibility and conversion, and new displays such as digital cinema, hi-vision
TV, flat displays and laser displays.

High-end servers- Establish standards for evaluating the performance of high-performance computers and servers, as well as forenergy-saving features and electrical properties.
- Push forward with studies on the high-end server standards required for applications such as Internet data
centers, domain services and cloud computing.

High-end software anddigital virtualization

Establish standards for operating systems, Chinese software formats, document layout, new-type
databases, middleware, high-end software such as built-in type software, digital content management andprotection, digital storage and interoperability, union of multi-media codecs, computer graphic modeling andsimulation, environment data expression, virtual reality/augmented reality/mixed reality and digital content
aggregation delivery platforms.

Software service andnetwork added-value
services

Establish standards for the plan, design and supervision of information system projects, governance andmanagement of IT outsourcing, IT operation and maintenance, IT services, software services; added-value webservice standards for Internet content filtering, interoperability, privacy protection, development and use of
information resources, etc.

Wulianwang(a network that links
objects):- Establish and revise standards for the application of wulianwang to fields such as traffic, public safety,
agriculture, environmental protection and forestry.
- Establish key basic standards for wulianwang, and standards for sensor networks, automatic identification
technologies and unified resource identifiers.
- Use for RFID army standards.

Cloud computing

Establish standards for cloud computing terminology and reference models, cloud data management and storage,
cloud platform interfacing and virtualization, next-generation search engines, next-generation network operating
systems, mass storage systems, and intelligent mass data

Source (in simplified Chinese) at http://images.mofcom.gov.cn/lczx/accessory/201201/1325841074090.pdf from p. 14 onwards
IC: integrated circuit
MEMS: microelectromechanical systems
ID: identification
RFID: radio frequency ID
IPv6: Internet Protocol version 6

General Administration of Quality Supervision, Inspection and Quarantine of the PRC
National(AQSIQ)
standards
Certification and Accreditation
Standardization Administration of China
Administration of China(SAC)(CNCA)

Industry
standards
The central government’s
departments in charge ofindustry

Local
standards
Standards section of eachregion’s Bureau of Quality andTechnical Supervision

Fig. 1. Government bodies responsible for standards in China.

NTT Technical Review

Global Standardization Activities

All-member meeting

Board of Directors

Technical Committees

Specialist
Committee

TC1: IP and Multimedia
Communications
TC3: Exchange Networks
TC4: Communication Power &Station Working Environment
TC5: Wireless Communications
TC6: Transmission Networks andAccess Networks

Management
Committee

Secretariate

ST2: Energy Conservation and Overall Useof Communications Facilities
ST3: Emergency Communications
ST4: Development and Use ofCommunication Infrastructures
Special teams

TC7: Network Management andOperation Support
TC8: Network and Information
Security
TC9: Electromagnetic Environment
and Safety Measures
TC10: Ubiquitous Networks
TC11: Mobile Internet Applications
and Terminals

Fig. 2. Organizational structure of CCSA.

membership is divided into full members, affiliate
members, and observers. In recent years, the numberof members has been on the rise, and in 2012 there
were 7 affiliate members, 29 observers, and 1 specialist member in addition to 279 full members.
In addition to carriers and vendors, CCSA’s standardization activities revolve around CATR (China
Academy of Telecommunication Research of
MIIT)––China’s only government-funded institution
for research on telecommunications. CATR is
engaged in providing support for government measures on communications as well as consulting andcertification services. At the same time, it is actively
involved in CCSA’s domestic standardization activities carried out by Technical Committees (TCs) andin international standardization activities carried out
by ITU-T.

Vol. 11 No. 11 Nov. 2013

5.2 O
 rganizational structure and work on
domestic standardization
At CCSA, TCs have been established in ten fields.
They are working on various standardization efforts
ranging from optical access to IPTV (Internet Protocol television), IPv6 (Internet Protocol version 6),
and the Internet of Things (IoT). In addition to TCs,
Special Teams (STs)––study groups on specific subjects––have also been set up. The CCSA’s organizational structure is shown in Fig. 2, and the areas thateach TC’s Working Group (WG) is focusing on arelisted in Table 2.
In 2013, certain fields were identified as key areas.
These were: Broadband China, TD-LTE, cloud computing, IoT, Internet datacenters, SDN (softwaredefined networking), next-generation satellite communications, information safety, and industry regulations. In terms of numerical targets, 400 projects intotal are to be completed, including national standards,Global Standardization Activities

Table 2. Areas of focus for TC WGs.
Study group

Subject studied

Study group

WG1

IP networks and equipment

WG1

Wireless communication management

WG2

IP services and applications

WG2

WG3

Content protection

Transmission network and accessnetwork management

WG4

New technologies and international
standardization

WG3

General

WG1

Wired network security

SWG2

IPTV

WG2

Wireless network security

WG1

Networks in general

WG3

Security management

WG2

Signaling protocols

WG4

Security infrastructure

WG4

Services and applications

---

---

WG1

Magnetic coupling of telecommunications
facilities

WG3

Wireless LAN and wireless access

WG4

cdmaOne/CDMA2000

WG5

3G security and encryption

WG6

Beyond3G

WG8Frequency

WG9

TD-SCDMA/WCDMA

WG10Satellite/microwave communication

WG1

Transmission networks

WG2

Access networks

WG3

Optical fiber/optical cables

WG4

Optical devices

TC1

TC3
TC4

TC5

TC6

industry standards, association standards, andresearch reports.
In recent years, China has pushed forward withtechnological development concerning its wulianwang*2 and Broadband China, treating this development as a national policy. Technological standards inthese fields are also being studied intensively.
Wulianwang is the subject of wide-ranging studies,from the distribution of sensor information to theexchange of information between smart city platforms. The TC10 Technical Committee on ubiquitous
networks, established in 2010, has been actively
working on the standardization of sensor networks,
M2M (machine-to-machine) communications, andsmart cities. The wulianwang that China aims tocreate will leverage the necessary applications toenrich people’s lives, which places the focus of discussions on the study of applications. Furthermore,
Broadband China includes the study of FTTH andIPv6, and CCSA’s involvement in IPv6 is especially
active, with initiatives such as the management of theworld’s largest IPv6 network.
In recent years, standardization work has intensified further, with a total of 646 drafts completed in


TC7

TC8

TC9

TC10

TC11

Subject studied

WG2

Anti-thunderstorm damage measures fortelecommunication systems

WG3

Electromagnetic radiation and safety

WG1

GeneralWG2Applications

WG3

Networks

WG4Development of technology andstandardization

WG1

GeneralWG2Service platforms and their applications

WG3

Terminals

TD-SCDMA: time-division synchronous CDMAWCDMA: wideband CDMA

2012 (national standards, industry standards, technical reports on communication standards, and research
reports). Also, documents in the communication
technology field drafted over the ten years since the
CCSA’s establishment include 2,663 industry standards, 506 national standards, 96 technical reference
documents, and 115 technical reports.
At the NTT Group, observers from NTT are participating in TC10, and observers from NTT DOCOMO are taking part in TC5, TC10, and TC11, gathering information on the progress of communications
standardization in China.
5.3 C ooperation with international standards
institutions
CCSA is actively engaged in cooperative initiatives
with international standardization institutions andstandardization conferences. For example, one canget an idea of the significance of its role from thenumber of articles contributed to international*2 Wulianwang: a network linking objects. The concept is similar tothat of the Internet of Things (IoT); China’s wulianwang, however, is used as a keyword to indicate a wider scope than IoT.

NTT Technical Review

Global Standardization Activities

standardization bodies over the ten years up to 2012,
which is in excess of 40,000; reportedly, the acceptance rate for the recommendation is also rising eachyear. CCSA is also trying to widen the scope of its
activities, for instance, by proposing the launch ofvarious new Focus Groups (FGs) in ITU-T. China’srole as a global leader in standardization is beginningto become apparent; over 40 key personnel are reported to have been appointed at international standardization organizations, and several hundred Chinese
representatives are working as editors of standards.
China has made its presence especially felt in the
ITU-T’s FG-M2M field, where it has proposed thelaunch of an FG and obtained the chairman’s appointment. The FG-M2M field is related to wulianwang,
which is the focus of much development in China.
In addition, China is actively involved in oneM2M,
a new M2M standardization body whose establishment was agreed upon by six standardization bodies:
ARIB (Association of Radio Industries and Businesses), ATIS (Alliance for Telecommunications
Industry Solutions), CCSA, ETSI (European Telecommunications Standards Institute), TIA (Telecommunications Industry Association), and TTA (Telecommunications Technology Association), and infact hosted a oneM2M conference in Beijing in
December 2012 [4].
China has also formulated a large number of proposals in oneM2M; at the 2nd conference held in
December 2012, it submitted 12 contributions,
including China’s standard Ubiquitous Network Terminology. This series of initiatives shows China’s
intention to lead discussions outside of the conventional framework of standardization bodies.

6. Conclusion
Despite the effects of various internal and external
issues, China’s economy is continuing to grow steadily, and remarkable developments are being achievedin the ICT field. In particular, a variety of systematic
initiatives such as the mid- to long-term development
plans for 2006–2020 and the 12th Five-Year Guideline are being pushed forward in the fields of scienceand technology in order to achieve technologicalinnovations originating from China. The same goesfor standardization. China is focusing particularefforts on standardization in the communications
field and is making vigorous advances: in 2012, 44
out of 211 standards in the 19 industrial fields established by the MIIT were communications-related,accounting for 21% of the total.
At the NTT Beijing Representative Office, we willcontinue to gather information on standardization
trends in the Chinese telecommunications industry on
behalf of the NTT Group, and to pursue and promote
new cooperative relationships with China.
References
[1]

[2]
[3]
[4]

D. Ikegami, “NTT Beijing Representative Office,” NTT Technical
Review, Vol. 11, No. 7, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=ntr2013
07ov1.html
PCT Yearly Review, 2013.
http://www.wipo.int/portal/en/news/2013/article_0021.html
China Communications Standards Association.
http://www.ccsa.org.cn/english/
CCSA, oneM2M Conference (in simplified Chinese).
http://www.ccsa.org.cn/worknews/content.php3?id=2992

Daisuke Ikegami

Associate Manager, NTT Beijing Representative Office.
He received the B.E., M.E., and Ph.D. degrees
from Waseda University, Tokyo, in 2002, 2004,
and 2007, respectively. Since joining NTT Service Integration Laboratories in 2007, he has
been engaged in research on IP service network
engineering. He is currently engaged in promotional activities for R&D technologies at NTT
Beijing Representative Office. He received theYoung Engineer Award from the Institute ofElectronics, Information and Communication
Engineers (IEICE) in 2010. He is a member of
IEICE.

Vol. 11 No. 11 Nov. 2013



External Awards
IEICE ELEX Best Paper Award in the Year 2012
Winners: Yoshio Takahashi†1 and Tsutomu Matsumoto†2†1	 System Platforms Sector, NTT DATA Corporation†2	Graduate School of Environment and Information Sciences, Yokohama National University

Date: September 18, 2013
Organization: The Institute of Electronics, Information and Communication Engineers (IEICE)
For “A Proper Security Analysis Method for CMOS Cryptographic Circuits”.
Differential Power Analysis (DPA) aims at revealing secret keys incryptographic devices by analyzing their power consumption as sidechannel information. Although power consumption models based ontransition probability were used to evaluate DPA resistance in previous studies, the adequacy of these models has not been adequately
confirmed. In this paper, we describe two experiments on obtaining
precise information of power consumption, and show that Random
Switching Logic, one of the DPA countermeasures, is in reality notsecure against DPA.
Published as: Y. Takahashi and T. Matsumoto, “A Proper Security
Analysis Method for CMOS Cryptographic Circuits,” IEICE Electronics Express, Vol. 9, No. 6, pp. 458–463, 2012.



NTT Technical Review