NTT Technical Review
May 2016 Vol. 14 No. 5
Feature Articles: Artificial Intelligence Research Activities in
NTT Group
Artificial Intelligence Research Activities and Directions in the NTT Group
Natural Language Processing Supporting Artificial Intelligence
Cloud-based Interaction Control Technologies for Robotics Integrated
Development Environment (R-envTM)
The Eyes as an Indicator of the Mind A Key Element of Heart-Touching-AI
Business Transformation Using Artificial Intelligence at NTT Communications
Artificial Intelligence Technology Development and Its Practical Use at
NTT DATA
Utilization of Artificial Intelligence in Call Centers

Regular Articles
Optical Switches Using Beam Steering by Computer Generated Hologram
Edge Router System that Distributes Traffic Flexibly According to Services

Global Standardization Activities
Standardization Activities at W3C TPAC 2015

External Awards/Papers Published in Technical Journals
and Conference Proceedings
External Awards/Papers Published in Technical Journals and Conference
Proceedings

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Artificial Intelligence Research
Activities and Directions in
the NTT Group
Takeshi Yamada, Satoshi Takahashi, Futoshi Naya,
Takashi Ikebe, and Shigeto Furukawa
Abstract

The research and development of artificial intelligence (AI) at NTT is advancing along four directions:(1) Agent-AI for analyzing information issued by people and understanding intentions and emotions inthat information; (2) Heart-Touching-AI for analyzing unconscious and unnoticeable aspects of a person’s mind and body and understanding deep psychological, intellectual, and instinctual states in that
person; (3) Ambient-AI for analyzing and understanding just about anything in the world (objects, people, the environment) and instantaneously predicting and controlling those things; and (4) Network-AI
for organically connecting and cultivating multiple types of AI and optimizing the entire social system.
This article introduces the technologies supporting these four forms of AI and specific AI-related activities in the NTT Group.
Keywords: artificial intelligence, machine learning, brain science

1. Introduction
We are now experiencing another artificial intelligence (AI) boom as interest in its possibilities continues to grow in science, business, and the mass media.
There are various reasons for this renewed interest in
AI. First, research in deep learning, a type of machinelearning technology, is progressing and coming todominate a variety of information-communication
fields such as image/voice recognition and understanding [1]. Also, when IBM’s Watson question andanswer (Q&A) system beat human champions in a
television quiz show, it ignited a trend in the use of
AI-related technology in the business domain. Finally, the commercialization of various types of humanoid robots has helped to make many people more
comfortable about a society in which people androbots coexist. In addition, some researchers predictthat a technological singularity in which AI comes tosurpass all human ability will occur by 2045 [2]. Thispossibility is generating a great deal of interest
1

throughout the world.
While it is important to get on board this AI trend
without delay and take a lead in advancing the
research on it, it is also important to analyze both thetechnological and business aspects of AI in a calm
and thorough manner. For example, deep learning
and Q&A systems constitute only a portion of technologies that will be needed in order to make people
more comfortable with AI, while in the businessdomain, the first signs that AI may bring about arevolution in the industrial structure are starting toappear.
In these Feature Articles, we introduce some of theactivities underway in the NTT Group that are
expected to lead to a genuine implementation of AI.
As an introduction to these articles, we explain here
the four directions that NTT is taking in the researchand development (R&D) of AI.

NTT Technical Review

Feature Articles

Agent-AI

Ambient-AI

Analyzing information issued bypeople and understanding intentions
and emotions in that information

Analyzing and understanding just about
anything in the world (objects, people,
the environment) and instantaneously
predicting and controlling those things

Personto
MachineAnalyzing unconscious and unnoticeable
aspects of a person’s mind and body aswell as understanding deep psychological,
intellectual, and instinctual states in that
personAI

Machineto
MachineOrganically connecting and cultivating
multiple types of AI and optimizing theentire social system

Heart-Touching-AI

Network-AI

Fig. 1. Four AI directions set by NTT.

2. NTT’s four AI directions
The world of AI is quite diverse with many andvaried supporting technologies, and people often
view AI in different ways. For example, AI is commonly thought of as the pursuit of human-like behavior in a computer, so that the computer, for all practical purposes, appears to be human. However, intoday’s information and communication technology
society, all sorts of devices and sensors are being connected to the Internet to form the Internet of Things(IoT), and the idea that these things can have some
type of intelligence and behave in an organized manner can also be called AI.
With this in mind, NTT has established the following four concepts or directions reflecting different
forms of AI that can be useful to society (Fig. 1).
(1)	Agent-AI for analyzing information issued bypeople and understanding intentions and emotions in that information. If we consider Q&AVol. 14 No. 5 May 2016

type AI to be a typical example of conventionalAI, then Agent-AI would be located far out on
the development line extending from that.
Agent-AI can also be called AI in the pursuit ofhuman-like behavior.
(2)	Heart-Touching-AI for analyzing unconscious
and unnoticeable aspects of a person’s mind andbody as well as understanding deep psychological, intellectual, and instinctual states in that
person. It is thought that deepening our knowledge of the mechanisms underlying the humanbrain and mind will lead to a new world in whichpeople and AI can interact in a more direct manner.
(3)	Ambient-AI for analyzing and understanding
just about anything in the world (objects, people, the environment) and instantaneously predicting and controlling those things. This type of
AI would analyze information from things andthe environment in addition to people and feed2

Feature Articles

Supporting intelligent tasks such as
diagnosis and brainstorming

Idea creation

Empowering users by assisting in daily
activities such as transportation andcommunication

Supporting and replacing routine tasks suchas contact center and reception counter
work and nursing care
Healthcare

RECEPTION

Mobile personal agent

Telepresence robot

Agent-AI

Images, voices, andother sensor data

Real world structuring
technology

Human understanding
technology
Understanding
Knowledge
exploration
Representation

Smartphones and robots
as user interfaces

Organized knowledge(Knowledge bases, rules)

Interaction technology

Fig. 2. Agent-AI.

the results of analysis back to the real world.
(4)	Network-AI for organically connecting and cultivating multiple types of AI and optimizing theentire social system. Reassessing the network as
an infrastructure from an AI perspective shouldmake it possible to create a completely newsocial system.
2.1 Agent-AI
As the name implies, Agent-AI is aimed at achieving advanced interaction with human beings as apersonified intelligent agent typified by a robot. This
would be accomplished in diverse ways such as byunderstanding people and the conditions surrounding
them through voice, language, images, and othertypes of media (human understanding technology),
by performing multimodal interaction that includes
human facial expressions, body language, and gestures (interaction technology), and by being conversant in multiple languages and performing compli3

cated reasoning based on a huge amount of knowledge (real world structuring technology) (Fig. 2).
One example of the above within the NTT Group is
the development of an interactive system that ismeant to achieve natural conversation with a computer. This is a joint project between NTT and NTT
DOCOMO, the latter of which is already providing
the Shabette Concier (Talking Concierge) voiceagent service for smartphones [3]. Further employing
the above technologies and exploiting all relevant
data such as the user’s profile and vital signs data canenhance contact center operations and customer service operations by partially automating the customer
reception process and by providing staff with useful
information in a timely fashion according to the circumstances. The main point, however, would be todevelop AI that could understand not only the user’s
intentions but also the user’s emotions and that would
always create a positive impression in the user. Similarly, in the healthcare field, AI technology could be
NTT Technical Review

Feature Articles

Education

Decoding and influencing
mental, neurological,
and muscle activity

Entertainment

Parenting and nurturing

Health and beauty

Healthcare and counselingSportsFig. 3. Heart-Touching-AI.

used to perform first-order processing of huge volumes of vital signs data, thereby reducing the doctor’s diagnostics testing load. In addition, intelligent
AI support could be used to ease the burden on nursing staff.
In this way, AI will come to replace or support certain human activities, but at the same time, AI andpeople will coexist and co-create sustainable societies
by doing those tasks that each does best. This process
will help to enrich people’s daily lives. In the future,
the aim will be to achieve an agent that can serve as a
background assistant throughout a person’s life. Suchan agent, for example, could enhance the skills andknow-how of young people in their 20s by offering
specialized advice, enhance the creative skills of
experienced staff in their 40s by supporting idea generation, and supplement the physical faculties andmemory power of senior citizens to improve theirlabor force availability and achieve a sustainable
workforce. The end result would be a brighter society
for both younger and older generations of people.
NTT’s current efforts in Agent-AI research andbusiness development are described in separate articles in this issue [4–8].
2.2 Heart-Touching-AI
As described above, Agent-AI is a form of AI that
supports people by providing them with knowledge
and functions. However, no matter how advanced theVol. 14 No. 5 May 2016

knowledge possessed by AI and how complex thereasoning that it performs, there are still doubts as towhether AI can become human-like in character.
Against this background, NTT aims to achieve what
it calls Heart-Touching-AI (HT-AI) that further pursues human-like characteristics by understanding themechanisms of human cognition through research inbrain science (Fig. 3).
HT-AI does not restrict itself to intelligence. It istargeted at understanding the essential and fundamental components of a person—the intellect,
instinct, and physical body—and to encourage that
person and expand his or her abilities. The idea here
is to break down emotional barriers between individuals and between an individual and society or theenvironment and to overcome physical barriers as
well, and to create a comfortable human society. In
other areas, too, such as sports and the arts, HT-AI isaimed at enabling anyone to experience the sensations and flashes of inspiration that only professional
athletes or accomplished artists are thought to have.
Inside the human brain, the neocortex is the outermost region and the most recent to appear in the evolutionary process. This region is mostly responsible
for rational and analytical thought and language functions, which is generally the pursuit of Agent-AI.
Underneath the neocortex, there are the cerebral limbic system, brain stem, and cerebellum, which are
older parts of the brain. These parts cannot be ignored
4

Feature Articles

Real-time cyber-physicalsystemDecode hidden cause and meaningof events from sensory information.

Decoding real-time
sensory information

DecodeDesigning intelligent
systems

Exploring optimal
scenarios
DesignSpatio-temporal
prediction

ExploreCausal reasoning

Anomaly detection
Explore optimal scenarios bysimulation incorporated with prediction,
reasoning, and detection.

Design and execute proactive
navigation and control in smart cities.

Fig. 4. Ambient-AI.

if we are to understand or influence unconscious cognitive processing, physical movements, and humancharacteristics such as trust, loyalty, likes/dislikes,
amiability, and motivation. These functions andmechanisms, however, are still not sufficiently understood, and measurement and evaluation techniques
are still being developed.
HT-AI is therefore a form of AI that aims to decodeor influence the state of human brain activity. As thename implies, HT-AI attempts to touch the heart. It
aims to provide the individual with a comfortable
existence rather than simply being useful. The road toachieving HT-AI and enabling people to enjoy its
benefits is long, but NTT has already begun activities
toward its realization. These efforts are described in a
separate article in this issue [9].
2.3 Ambient-AI
Ambient-AI pursues things that have some level ofintelligence and that behave in an organized manner.
The word ambient is similar in meaning to atmosphere and environment. For example, ambient music
5

is meant to create an atmosphere in the listener’s
mind. In Ambient-AI, things that exist just about
everywhere in the environment—that is, devices having sensors and actuators—possess intelligence.
Interconnecting them in a network so that they cancommunicate and interface with each other will
enable them to make decisions and act autonomously
and to support people in decision-making (Fig. 4). In
2006, NTT proposed the concept of ambient intelligence, in which the existence of a thing with intelligence can be likened to a fairy or guardian angel“who normally remains hidden while quietly watching over a person but comes to the rescue whenever adifficulty or emergency arises in that person’s life”[10]. In the ten years or so that have passed sincethen, the cloud, big data, and mobile communications
have made great inroads in society, and significant
progress has been made in IoT and machine learning.
It can also be said that the elements essential toachieving ambient intelligence are finally being readied.
The ambient intelligence mentioned above includes
elements of Agent-AI as a personified agent. However,
NTT Technical Review

Feature Articles

Services

Comprehensive optimal control

Rapidly control a major
accident based on
judgments made in eacharea by analyzing volatile
big data.

Exchange knowledge
between AIs and redesign
the entire package
delivery plan.

Network

Re-optimize the entire
production plan by negotiating
between AIs for alternative
parts supply.

Infrastructure

Area
Accident

Area-centric
control
Automatic blocking of unauthorizedaccess through inter-network
cooperation

Network facilities detect
the landing of an
emergency medical
helicopter and then movethe drones away.

Autonomous facility
add/remove process

Maintenance-free
networks

Testable and inspectablemachine learning

No-call-drop
wireless network

Fig. 5. Network-AI.

the emphasis in Ambient-AI is a real-time cyberphysical system that continuously repeats the three
processes of decoding hidden causes of events in thereal world through intelligent real-time sensing,
exploring optimal scenarios by performing prediction, reasoning, and detection using the cloud environment, and providing feedback based on an appropriate system design. In this way, Ambient-AI will beable to measure the real world (the physical component) in real time through devices and sensors situated ubiquitously in the environment. It will also beable to predict when, where, and what will occur inthe near future by combining the above measurements with information on the Internet (the cybercomponent), to infer cause-and-effect relationships,
to detect hidden signs with high accuracy, and basedon simulations that incorporate the above, to searchout and establish optimal scenarios and perform proactive control. NTT is researching and developing
spatio-temporal multidimensional data analysis
technology that models space-time relationships andpredicts where and when a phenomenon will occur[11].
Vol. 14 No. 5 May 2016

2.4 Network-AI
If Ambient-AI evolves even further, it can be envisioned that devices, people, and services connected toa low-latency, high-throughput network will become
connected and linked as needed to all sorts of resources, thereby bringing about Network-AI, in which the
network itself becomes AI. As a result, large volumesof time and space specific data (volatile big data) will
be continuously generated at network terminal points
and all areas covered by the network. It is importantthat these data be used effectively (Fig. 5).
However, sharing all of these data in the network—
even in a high-performance network—is not efficient.
It is therefore necessary to process the data first in anarea-centric manner and to perform rapid execution
and control based on judgments made in each area,even if it is not a 100% perfect solution (area-centric
control). For example, in the event of a major disaster,
we can envision how network facilities in charge ofan affected area could detect the upcoming landing ofan emergency medical helicopter and then activate
the emergency control of drones already active in thatarea to rapidly move them away to avoid potential
6Feature Articles

collisions.
Next, after such processing in local areas has been
completed, essential portions of that processing could
be hierarchically shared over the network, enabling
comprehensive optimal control. For example, we canenvision how disaster information generated in certain regions could be used to redesign plans for
global delivery of relief supplies. In this way, it isimportant that a good balance be established between
area-centric control, which corresponds to processingin the cerebellum of the human brain, and comprehensive optimal control, which corresponds to the
cerebrum.
In addition, it is important that a high-quality, safe,
and secure network be provided in both normal andabnormal times by applying AI technology to the
network. NTT is already applying machine-learningtechnology for early detection of signs of network
failure and for operation automation and traffic prediction [12]. Future applications of AI may include
automatic blocking of unauthorized access throughinter-network cooperation, construction of maintenance-free networks, provision of a no-call-drop
wireless network, and implementation of an autonomous facility add/remove process. New machinelearning techniques will be required to ensure thesafety of these autonomous operations, in whichpeople can test and inspect what is being learned andacquired.
3. Future outlook
This article introduced NTT’s approach to AI infour key directions. These four forms of AI are in noway independent of each other but simply reflect different aspects of the broad world of AI. In any service, each of these four forms of AI will appear in one
way or another, and the technologies common tothem, such as machine learning, pattern recognition,and optimization, are not few in number. In addition,
R&D of Agent-AI, for example, is making good
progress, while that of the other forms of AI, forinstance, Network-AI, is still in its early stages. In
any case, NTT plans to further promote R&D in eachof these four AI directions going forward. The goal isto achieve an enriching and comfortable society inwhich AI supports people in their daily lives and AI
and people co-create sustainable societies by doing

7

what each does best.
References[1]

S. Araki, M. Fujimoto, T. Yoshioka, M. Delcroix, M. Espi, and T.
Nakatani, “Deep Learning Based Distant-talking Speech Processing
in Real-world Sound Environments,” NTT Technical Review, Vol. 13,
No. 11, 2015.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201511fa4.html[2] R. Kurzweil, “The Singularity Is Near: When Humans Transcend
Biology,” Penguin Books, 2006.
[3] K. Onishi and T. Yoshimura, “Casual Conversation Technology
Achieving Natural Dialog with Computers,” NTT DOCOMO Technical Journal, Vol. 15, No. 4, pp. 16–21, 2014.
https://www.nttdocomo.co.jp/english/binary/pdf/corporate/technology/rd/technical_journal/bn/vol15_4/vol15_4_016en.pdf[4] Y. Matsuo, R. Higashinaka, H. Asano, and T. Makino, “Natural Language Processing Supporting Artificial Intelligence,” NTT Technical
Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa2.html[5] T. Yamada and H. Yoshikawa, “Cloud-based Interaction Control Technologies for Robotics Integrated Development Environment(R-envTM),” NTT Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa3.html[6] K. Ito, S. Nishido, and T. Yamazaki, “Business Transformation Using
Artificial Intelligence at NTT Communications,” NTT Technical
Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa5.html[7] O. Shirotsuka, “Artificial Intelligence Technology Development and
Its Practical Use at NTT DATA,” NTT Technical Review, Vol. 14, No.
5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa6.html[8] S. Kawamura, K. Machida, K. Matsui, D. Sakamoto, and M. Ishii,“Utilization of Artificial Intelligence in Call Centers,” NTT Technical
Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa7.html[9] S. Furukawa, M. Yoneya, H.-I. Liao, and M. Kashino, “The Eyes as an
Indicator of the Mind—A Key Element of Heart-Touching-AI,” NTT
Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa4.html[10] E. Maeda and Y. Minami, “Steps towards Ambient Intelligence,” NTT
Technical Review, Vol. 4, No. 1, 2006.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr200601050.pdf[11] F. Naya and H. Sawada, “From Multidimensional Mixture Data
Analysis to Spatio-temporal Multidimensional Collective Data Analysis,” NTT Technical Review, Vol. 14, No. 2, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201602fa2.html[12] K. Ishibashi, T. Hayashi, and K. Shiomoto, “Improving Network
Management and Operation with Machine Learning and Data Analytics,” NTT Technical Review, Vol. 14, No. 2, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201602fa5.html

NTT Technical Review

Feature Articles

Takeshi Yamada

Executive Research Scientist, Research Planning Section, Machine Learning and Data Science Center, NTT Communication Science Laboratories.
He received a B.S. in mathematics from the
University of Tokyo in 1988 and a Ph.D. in informatics from Kyoto University in 2003. He joined
the Electrical Communication Laboratories at
NTT in 1988. He was a visiting researcher at the
School of Mathematical and Information Sciences, Coventry University, UK, from 1996 to
1997. He was a group leader of the Emergent
Learning and Systems Research Group from2006 to 2009 and an executive manager of Innovative Communication Laboratory from 2012 to2013 at NTT Communication Science Laboratories. His research interests include data mining,statistical machine learning, graph visualization,
metaheuristics, and combinatorial optimization.
He is a senior member of the Institute of Electrical and Electronics Engineers (IEEE) and theInstitute of Electronics, Information and Communication Engineers (IEICE), and a member ofthe Association for Computing Machinery andthe Information Processing Society of Japan(IPSJ).

Satoshi Takahashi

Executive Manager, Executive Research Engineer, Supervisor, Audio, Speech and Language
Media Project, NTT Media Intelligence Laboratories.
He received his B.E., M.E., and Ph.D. in information science from Waseda University, Tokyo,
in 1987, 1989, and 2002. Since joining NTT in
1989, he has been engaged in research on speech
recognition, spoken dialog systems, and patternrecognition. He is a member of the AcousticalSociety of Japan (ASJ) and IEICE.

Futoshi Naya

Senior Research Scientist, Supervisor, Innovative Communication Laboratory, NTT Communication Science Laboratories.
He received a B.E. in electrical engineering, an
M.S. in computer science, and a Ph.D. in engineering from Keio University, Kanagawa, in
1992, 1994, and 2010. He joined NTT Communication Science Laboratories in 1994. From2003 to 2009, he was with Intelligent Robotics
and Communication Laboratories, Advanced
Telecommunications Research Institute International (ATR). His research interests include communication robots, sensor networks, pattern recognition, and data mining in cyber physical systems. He is a member of IEEE, the Robotics
Society of Japan, the Society of Instrument andControl Engineers, and IEICE.

Vol. 14 No. 5 May 2016

Takashi Ikebe

Senior researcher, NTT Network Service Systems Laboratories.
He received his B.E., M.E., and Ph.D. in engineering from the University of Electro-Communications, Tokyo, in 2000, 2002, and 2008. He
joined NTT Network Service Systems Laboratories in 2002 and studied call control software,
middleware, operating systems, and deployment
scenarios. During 2006–2007, he was active indeveloping the Carrier Grade Linux specifications at OSDL (now part of Linux Foundation).
He has extensive research and product experience in Linux and software-based call control
systems. He received the 2006 OSDL Contribution Award. His recent research topics include
IoT, device computing, and service-enable network architectures. He is a member of IEICE.

Shigeto Furukawa

Senior Research Scientist, Supervisor, Group
Leader of Sensory Resonance Research Group,
Human Information Science Laboratory, NTT
Communication Science Laboratories.
He received a B.E. and M.E. in environmental
and sanitary engineering from Kyoto University
in 1991 and 1993, and a Ph.D. in auditory perception from University of Cambridge, UK, in 1996.
He conducted postdoctoral studies in the USA
between 1996 and 2001. As a postdoctoral associate at Kresge Hearing Research Institute at the
University of Michigan, USA, he conducted
electrophysiological studies on sound localization, specifically the representation of auditory
space in the auditory cortex. He joined NTT
Communication Science Laboratories in 2001.
Since then, he has been involved in studies on
auditory-space representation in the brainstem,
assessing basic hearing functions, and thesalience of auditory objects or events. In addition, as the group leader of the Sensory Resonance Research Group, he is managing various
projects exploring mechanisms that underlie
explicit and implicit communication between
individuals. He is a member of the AcousticalSociety of America, ASJ (member of the Executive Council), the Association for Research in
Otolaryngology, and the Japan Neuroscience
Society.

8

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Natural Language Processing
Supporting Artificial Intelligence
Yoshihiro Matsuo, Ryuichiro Higashinaka,
Hisako Asano, and Toshiro Makino
Abstract

Natural language processing technology is considered to be essential to achieving artificial intelligence (AI) that can substitute for humans in certain roles. Most robots and AI systems that appear infiction are equipped with a natural language interface, and it is expected that clever AI will learn on its
own from encyclopedias, news reports, and other textual sources. Exactly what kinds of AI functions will
be achieved through natural language processing technology? This article introduces the role that natural
language processing technology can be expected to play in achieving AI.
Keywords: artificial intelligence, natural language processing, knowledge processing

1. Introduction
In artificial intelligence (AI), a technical field thatis difficult to define, natural language processing
technology appears to be a major element. This isclearly demonstrated by the use of natural language
conversations to measure computer intelligence in theTuring test, which is a well-known technique fordetermining whether a certain machine is intelligent.
In measuring the intelligence of something in which
intelligence or intellectual level cannot be directly
observed, it is natural to adopt an approach in which
intelligence is assessed based on responses to queries,
since linguistic expression is surely a typical meansof exhibiting intelligence.
What kind of relationship does AI then have withnatural language processing? A schematic representation of the relationship between AI and natural
language processing is shown in Fig. 1. Agent-AI,
which is introduced in the feature article “Artificial
Intelligence Research Activities and Directions in the
NTT Group” [1] in this issue, is a form of AI that possesses abundant knowledge and a capacity for making judgments so that it can replace humans in certain
tasks and assist humans in their daily lives. Scenarios
in which Agent-AI would have a connection to natural language can be broadly divided into reading of
1

documents and interaction with the user. Thus,
Agent-AI achieved with natural language processing
would read documents extensively and accumulateknowledge, recognize user utterances, and make anappropriate response based on that accumulated
knowledge. We consider that the Turing test could bea means of measuring in some way whether Agent-AI
is producing effective responses.
In this article, we introduce how natural language
processing technology has so far been used in AI
industrial applications and the role it is expected toplay in the future.
2. Text-to-knowledge technology forunderstanding the meaning of text
If we assume that natural language expressions
demonstrate human intelligence, documents written
in natural language should then be filled with intelligent information. The primary role of natural language processing technology is to find informationthat is needed and beneficial from this text filled withintelligent information. Yet, text itself is no more thana collection of character strings, so evaluating data inthis form to find what is needed and beneficial cannot
be done. However, if text can be arranged in amachine-readable form through natural language
NTT Technical Review

Feature Articles

Understanding theuser’s speech

Understanding writtenmaterial by reading it(Intellectually) responding to
the user on the basis of theunderstood content

Fig. 1. Relationship between AI and natural language processing technology.

processing, information processing with a computational ability surpassing that of humans will become
possible. To this end, we can consider applications of
natural language processing to an informationsearching technique that looks for desired information from within a huge amount of information, or toa text mining technique that consolidates, arranges,
and presents information completely unreadable tohumans in its original form in a relatively short periodof time.
An example of applying the former technique is aquestion answering system [2]. A typical approachthat most people would take to learn about something
would be to search out a related section or entry froman encyclopedia, a textbook, or the web and retrieve
an answer from that source. This is the so-called
information-searching procedure, but if the sequenceof steps in this procedure could be automated, itwould be of great assistance in human activities.
Next, examples of applying the latter techniquewould be the mining of a contact-center log to analyze customers’ voices and sentiment analysis ofusers’ comments on the Internet [3]. While it may bepossible to uncover valuable information straightfrom the voices and comments of consumers, differentiating what is good and bad from such a hugeamount of information is not an easy task. Furthermore, such analysis would take time, so if this information was to be applied to some sort of decisionmaking, it could hardly be used for making rapid
decisions. Thus, the ability to analyze a large volume
of documents in a short period of time would be beneficial to corporations and organizations that need toVol. 14 No. 5 May 2016

conduct such tasks.
What kind of language processing technology istherefore needed to meet the above requirements?
That would be technology capable of extracting
information from documents written in natural language and organizing that information as knowledge.
We have undertaken the research and development ofrich indexing technology for extracting semantic
information from text and have been successful inextracting named entity expressions and opinion
information and in identifying the meaning of namedentities [3]. The automatic extraction of such information from text will enable the detection of answers
in retrieved documents and the tabulation of information from large volumes of text.
3. Utterance understanding technology for anatural language ICT interface
Another function expected of natural language processing technology is a computer interface based onnatural language.
Exchanges between humans and computers arenormally carried out according to a specialized communication format consisting of menus, icons, andcommands manipulated by a keyboard, mouse, orother device. This type of interface has been designedwith efficient computer operation in mind, and a user
who becomes proficient in its use can indeed perform
efficient and effective operations. However, communication with a computer can be quite difficult for
those uncomfortable with such an interface, and thisproblem is thought to be one of the factors giving rise2

Feature Articles

What herbicidal
method should
I use?

I hate digging up weeds!
There’ll be weeds all overthe place before you
know it!
How can I get rid of theweeds?
Pulling them out is a reallyhard job.
I really want to get rid of
the weeds.

It’s really tough when alot of weeds suddenly
spring up.
The use of weed barrier
fabric is recommended.

An awful lot of ragweed
has sprung up.

Fig. 2. Determining sameness between user questions and FAQ on paper.

to the so-called digital divide. In contrast, natural
language can be viewed as an extremely basic humancommunication medium. Of course, a certain level ofproficiency is also necessary to communicate withnatural language, but this is a basic ability that mostpeople come to possess through daily life and education.
The development of an information and communication technology (ICT) interface using natural language will create value in two ways. The first is thatpeople will be able to use ICT and enjoy its benefits
without having to develop special skills, and the second is that a standard means of communication
between human beings and computers in the form ofnatural language will enable people and AI to coexist
in society.
One example of the former is a voice agent. The
operation of a smartphone is still rather complicated,
and learning how to use its functions well is not thateasy. Furthermore, while much knowledge can beobtained by querying a search engine, a certain
amount of information technology literacy is neededin order to use a search engine effectively. Consequently, if questions could be posed to AI by voiceusing natural language, a further expansion of ICT
users and usage scenarios could be expected.
An example of the latter type of value creation is acustomer-service support terminal for use at a contact
center or retail store. At contact centers, customers
talk with operators by telephone. We can envision
how a computer could listen in on these conversations
and provide just the right information at just the righttime. This kind of capability could make solving customers’ problems more efficient. Similarly, at a retailstore, a small robot could be positioned next to a3

salesperson at a cash register or information counter.
When the salesperson was discussing products withcustomers, the robot could promptly offer relevant
brochures or documents when needed, thereby serving as a very capable assistant.
Common to the above examples is the need forutterance recognition technology. Several ways ofrecognizing an utterance can be considered. One
would be to translate natural language into a computer language such as Structured Query Language(SQL), which is commonly used for database searching [4]. If a natural language query could be translated and entered into the SQL search conditions
field, it would then be possible to search databases byusing natural language. Another way of recognizing
an utterance would be to judge the equivalence of anutterance in terms of its intention (Fig. 2). For example, let’s consider a frequently asked questions (FAQ)
search. An FAQ document consists of frequently
asked questions, where knowledge is represented bypairs of questions and answers (Q&A). In this search,a user poses some kind of question (query). However,the query rarely agrees with a question at the character string level, but if it can be determined that thequery has the same intention as a certain question, thecorrect answer can be provided.
4. Casual conversation technology for achieving
human-like dialogue
The technologies described so far serve to makehuman life much more efficient, but they are not sufficient in themselves to achieve Agent-AI. If a humanbeing and Agent-AI are to become true partners, theymust be able to trust and understand each other and
NTT Technical Review

Feature Articles

Judgment of user
desire to continue
the current topic

User

User utterance

Utterance
understanding
moduleSystemutterance
intention
System utterance

Utterance
conversion module

User information
extraction

Dialoguecontrol module

Topic/user
utterance
intention

Utterance
characterization
System utterance(after conversion)

Utterance
history

Utterance
generation module

User
information
database
Knowledge
database

Content on Internet

Utterance
database

Fig. 3. Configuration of casual conversation technology.

even share emotions. When two people meet eachother for the first time, they would hardly start talking
about work from the start. It is only after they havebecome more familiar with each other that theywould begin to talk freely. Furthermore, in times ofneed, partners should help, comfort, and emotionally
support each other. Agent-AI must also be equippedwith such functions. As is known from the Media
Equation [5] theory, a human being interacting with acomputer tends to behave as if the computer were
human. Thus, the more intelligent a computer
becomes, the more human in character it should be.
In human society, a casual conversation is often theoccasion for two people to get to know each other orto become emotionally closer.
With this in mind, NTT Media Intelligence Laboratories has been researching casual conversation technology [6]. This technology will enable a user toconverse with a computer in everyday language on
any topic. Casual conversation is, of course, easy forhuman beings but extremely difficult for computers.
This is because current technology cannot easily
handle the many and varied topics that a user may
want to talk about, and it cannot develop a deep
understanding in the way that a human can of complex subjects raised by the user. In the AI field, theformer problem is known as the frame problem andthe latter as the symbol grounding problem. Against
this background, we have undertaken the development of casual conversation technology and havesucceeded in achieving a certain level of casual conversation with a computer by combining large-scale
text data and language processing technology. This
technology is being used in casual-conversation
Vol. 14 No. 5 May 2016

application programming interfaces for developer use
and in commercially available communication toys.
A system based on casual conversation technology
structures large volumes of content on the Internet(blogs, microblogs) using language processing technology and constructs an utterance database andknowledge database (Figs. 3 and 4). When a user
utterance is input, the system recognizes the topic andintention of that utterance based on the context. It
then produces a response in line with the user’s intention by searching for an appropriate system utterance
from an utterance database or by generating a systemutterance using a knowledge database. If the user
utterance is a question, the system uses question
answering technology to obtain a word or phrasefrom the Internet as a response. This mechanism
enables the system to produce responses on a variety
of topics that may appear in casual conversation.
However, it is not sufficient in itself to achieve
human-like behavior. Achieving Agent-AI that canform an attachment as a partner and carry on extended conversations will require mechanisms for remembering things about the conversation partner, expressing individuality, and understanding emotions. For
this reason, we are equipping Agent-AI with techniques for user information extraction, utterance
characterization, and judgment of user desire to continue the current topic. These techniques will enable
the construction of user information databases basedon user utterances, conversion of system utterances toones expressing a personality, and a change of topic
if the system detects that the user appears to have lostinterest in the current topic.

4Feature Articles

Past user utterances
Ramen wa suupu ga inochi nannya.
(Translation: The broth is the key to ramen.)
Yes, I’d like totalk about that

You understandme

I completely agree.

Ramen wa honma ni umai waa.
Seyakedo, karada ni wa yoku nainya.
(Translation: Ramen is really tasty,
but it’s not healthy.)

Hmmm…

• Interests/Preferences:classical music• Residence: Yokosuka

I love classical music!
I live in Yokosuka.

I really want to listen
to music!
You love classical music,
don’t you?

System (Kansai obachan* character)*Middle-aged womanfrom Kansai

Are there any concerts
I can go to around here?

Sou ieba kinou Hanshin katta dee.
(Translation: Now I remember, the Hanshin
Tigers won the game yesterday.)

There’s going to be one in
Yokosuka soon.
SystemUsing user information extraction
technology enables system to remember
user preferences and converse with them.

Technologies for utterance characterizationand for judging whether to continue a topic enable
users to talk to characters about their favorite topics.
In the above dialogue example, words in blue areexpressions of Kansai dialect. The corresponding
English words are also shown in blue.

Fig. 4. Example of conversation enabled by casual conversation technology.

5. Future outlook
Finally, we would like to touch upon the Todai
Robot Project (Can a Robot Get into the University ofTokyo?), a major AI project that has been attracting
attention. This project is a grand challenge led by
Japan’s National Institute of Informatics to develop
AI that can pass the entrance exam of the Universityof Tokyo. Specifically, it aims to achieve a high scorein the National Center Test for University Admissions
by 2016 and to pass the University of Tokyo exam by2021. Passing the latter will require the developmentof AI that can take and pass tests in a variety of academic disciplines.
NTT has been participating in this project since2014 and has been placed in charge of English as atarget discipline [7]. In addition to learning vocabulary and grammar, English presents a variety of problems, including the reading and comprehension oflong sentences and conversational text, the matching
up of text with tables and figures, and the use ofdiverse language processing skills. Solving problems
that human beings actually solve is what achieving
Agent-AI is all about. The program that we have constructed for solving these English problems can presently achieve a level of performance equivalent tothat of an average human examinee. Our aim, however, is to achieve Agent-AI that is useful in problem
5

solving and supportive to people through the experiences gained in this project and the results ofresearching and developing casual conversation technology.
References
[1]

[2]

[3]
[4]

[5]
[6]

T. Yamada, S. Takahashi, F. Naya, T. Ikebe, and S. Furukawa, “Artificial Intelligence Research Activities and Directions in the NTT
Group,” NTT Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa1.html
R. Higashinaka, K. Sadamitsu, K. Saito, and N. Kobayashi, “Question
Answering Technology for Pinpointing Answers to a Wide Range ofQuestions,” NTT Technical Review, Vol. 11, No. 7, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201307fa4.html
Special Feature: “Natural Language Processing Technologies for
Portal Services,” NTT Technical Review, Vol. 6, No. 9, 2008.
https://www.ntt-review.jp/archive/2008/200809.html
K. Sadamitsu, J. Shimamura, G. Irie, S. Tarashima, T. Yoshida, R.
Higashinaka, H. Nishikawa, N. Miyazaki, Y. Ijima, and Y. Nakamura,“Media Processing Technology for Achieving Hospitality in Information Search,” NTT Technical Review, Vol. 13, No. 4, 2015.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201504fa5.html
B. Reeves and C. Nass, “The Media Equation: How People Treat
Computers, Television, and New Media Like Real People and Places,”
CSLI Publications and Cambridge University Press, 1996.
R. Higashinaka, K. Imamura, T. Meguro, C. Miyazaki, N. Kobayashi,
H. Sugiyama, T. Hirano, T. Makino, and Y. Matsuo, “Towards an
Open-domain Conversational System Fully Based on Natural Language Processing,” Proc. of COLING 2014 (the 25th International
Conference on Computational Linguistics), pp. 928–939, Dublin,
Ireland, Aug. 2014.

NTT Technical Review

Feature Articles[7]

R. Higashinaka, H. Sugiyama, H. Isozaki, G. Kikui, K. Dohsaka, H.
Taira, and Y. Minami, “Taking the English Exam for the ‘Can a Robot
Get into the University of Tokyo?’ Project,” NTT Technical Review,Vol. 13, No. 7, 2015.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201507ra2.html

Yoshihiro Matsuo

Hisako AsanoGroup Leader, Senior Research Engineer,
Supervisor, Audio, Speech, and Language Media
Laboratory, NTT Media Intelligence Laboratories.
He received a B.S. and M.S. in physics from
Osaka University in 1988 and 1990. He joined
NTT Communications and Information Processing Laboratories in 1990. He moved to NTT
Cyber Space Laboratories (now NTT MediaIntelligence Laboratories) in 2001. His researchinterests include multimedia indexing, information extraction, and opinion analysis. He is amember of the Information Processing Society ofJapan (IPSJ) and the Association for Natural
Language Processing (NLP).

Senior Research Engineer, Audio, Speech, andLanguage Media Laboratory, NTT Media Intelligence Laboratories.
She received a B.E. in information engineeringfrom Yokohama National University, Kanagawa,in 1991. She joined NTT Information Processing
Laboratories in 1991. Her research interests
include natural language processing, especiallymorphological analysis, and information extraction. She is a member of IPSJ and NLP.

Toshiro Makino

Ryuichiro Higashinaka

Senior Research Scientist, Audio, Speech, andLanguage Media Laboratory, NTT Media Intelligence Laboratories.
He received a B.A. in environmental information, a Master of Media and Governance, and a
Ph.D. from Keio University, Kanagawa, in 1999,
2001, and 2008. He joined NTT in 2001. His
research interests include building question
answering systems and spoken dialogue systems.
From Nov. 2004 to Mar. 2006, he was a visiting
researcher at the University of Sheffield in the
UK. He received the Maejima Hisoka Award
from the Tsushinbunka Association in 2014. He
is a member of the Institute of Electronics, Information and Communication Engineers, the Japanese Society for Artificial Intelligence, IPSJ, and
NLP.

Vol. 14 No. 5 May 2016

Senior Research Engineer, Supervisor, Audio,
Speech, and Language Media Laboratory, NTT
Media Intelligence Laboratories.
He received his B.E., M.E., and Dr. Eng. inelectrical engineering from the University ofTokyo, in 1987, 1989, and 1992. He joined NTT
Network Information Systems Laboratories in1992. He is currently a senior research engineer,
supervisor at NTT Media Intelligence Laboratories and is engaged in the research and development of natural language processing.

6

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Cloud-based Interaction Control
Technologies for Robotics Integrated
Development Environment (R-envTM)
Tomohiro Yamada and Hiroshi Yoshikawa
Abstract

Communication robots are able to detect the surrounding situation through cameras or sensors, andthey use motion to show non-verbal expressions. Furthermore, automobiles and personal devices suchas vacuum cleaners are being robotized with the application of verbal communication technology. In thenear future, it will become very important for various robots to be able to connect to and work together
with devices that surround us in order to support human activities and enhance our lives. This article
introduces R-envTM, a cloud-based Interaction Control technology.
Keywords: multi-modal interaction, cloud robotics, networked robot

1. History of robots and artificial intelligence
The word robot was reportedly first used by Czech
playwright Karel Čapek in his 1920 science fiction
play “Rossum’s Universal Robots” [1]. The robot’s
role in the play was to replace some human labor.
Čapek’s robot would exert physical power or intellectual power far beyond that of humans according to
the task intended. In one scene of the play, a highly
intellectual robot with conscious awareness communicates with humans. In the literary and theatricalworld, robots demonstrate human-like physical and/or intellectual capabilities that in some cases exceed
those of humans. The role of robots in society is aquestion often asked since the early days of the introduction of robots. In the technology field, robots andartificial intelligence (AI) mutually influence eachother’s development.
Some key events in the development of robots
include the establishment of the Robotics Society ofJapan in 1983 and the first International Conferenceon Robotics and Automation in 1984 organized by
the Institute of Electrical and Electronics Engineers(IEEE). In 1986, the Japanese Society for Artificial
Intelligence was established, and in 1988, NTT
launched the Human Interface Laboratory Intelligent
1

Robot Research Project with a mission to introduce
robot control technology into production sites inJapan. Thus, research commenced on computer
vision, motion control, and methods of training
robots to understand the goals of the work assigned tothem, otherwise known as robot-teaching technology[2].
Technological development has been centered on
industrial robots until fairly recently. However,research on the interaction between humans androbots has advanced a great deal and is now quite
active. In 1991, IEEE established RO-MAN (International Symposium on Robot and Human Interactive
Communication) to focus on communication between
humans and robots.
In 1999, Sony Corporation launched AIBO, a dogtype pet robot, which made it possible for the generalpublic to actually see and experience high level communication robots that until then had been limited toresearch laboratories. AIBO’s algorithms and sensors
enable it to understand the intention of users and to
express its reactions by using light, sound, andmotion [3].
Honda Motor Co., Ltd. in 2000 announced ASIMO,
a humanoid robot that was able to walk like a human[4].
NTT Technical Review

Feature Articles

2. NTT’s development of robotic technology
NTT began researching and developing a network
robot platform to permit the collaboration of multiple
robots over the network. Part of this work was conducted under contract from the Ministry of Internal
Affairs and Communications (MIC) of Japan as“Comprehensive research and development of network-human interface (network robot technology)”
from 2004 to 2008. Network robot platform technology research was conducted with one aim being torealize a novel networked society where various services could be used anywhere, any time by connecting sensors, appliances, and robots, thereby enablingthem to work in collaboration over a highly advanced
network [5].
Three types of robots were developed:(1)	A tangible type of humanoid robot called avisible robot(2)	A virtual robot, which was a software agent on
appliance software(3)	An unconscious robot consisting of sensors inthe surrounding environment
We also developed a mechanism enabling these different types of robots to be connected and to work incollaboration by exchanging necessary information
over the network.
To enable various kinds of information to beexchanged between different types of robots manufactured by different companies, we devised a
description method to handle core information related to the user (4W: who, when, where, and what ofactions and behavior) and core information related to
the robot (4W1H: who, when, where, what (its capabilities are), and how the robot is behaving).
A network robot platform that incorporates thistechnology makes it possible to integrate robots thatwere previously difficult to connect to each other. We
conducted an experiment on connecting robots andtransferring robot relationship information using four
existing description methods over a standard communication protocol.
3. Novel robot collaboration service to expand
human potential
As previously described, different types of robots
are being released by numerous companies, and they
are starting to be experimentally used for various
purposes. Furthermore, we are seeing increased
sophistication and robotization of appliances andvehicles such as cleaning robots and self-driving cars,
Vol. 14 No. 5 May 2016

which were not developed as imitations of humans or
animals. Therefore, it can be assumed that the variety
of intelligent robots will only increase.
As reported by robot start inc. [6], the number ofcommunication robots in Japan will reach 2.65 million by the year 2020. In view of the fact that there areapproximately 55 million households in Japan [7],this number suggests there will be one communication robot for every 20 households. This number islikely to increase, judging from the number of cleaning robots, which are only now coming into wide use.
What benefits can we expect from intelligent robots
with AI? Are the benefits limited to increasing labor
efficiency as evidenced by industrial robots or as in
the play written by Karel Čapek?
The expression support of human growth is the key
to new services that we are trying to achieve at NTT
Service Evolution Laboratories. We are making
efforts to develop a novel service that will enablerobots and machines to understand the situation thatsurrounds us, encourage us, enhance our ability, andhelp us perform new actions.
For instance, with regard to children, the new service will watch over a child, support the child in communicating with others, enhance the child’s learning
motivation, become the child’s good friend, and last
but not least, grow up with the child. With regard to
the elderly, the new service will enable robots andmachines to provide encouragement and support in
healthcare and external activities, improve social participation, and support an independent daily life.
For adults who are in their prime, the new service
will enable robots and machines to help adults with
their daily schedule and task management and support them in their sporting activities so that they canspend their time happily and efficiently. We aim todevelop robots that will learn and mature with their
human partners over an entire lifetime. However, thisdoes not mean we must continue to use the samerobot hardware forever. The robot will store information about the user’s preferences and characteristics
on the cloud. It will also have a brain so that it canlearn and mature with its user. The hardware of therobot may change from time to time or with a change
in location. There may even be two or more devices.
However, by connecting these robots and devices, we
can overcome spatial and temporal distance andachieve a robot that can support human activity overhis/her entire lifetime.

2

Feature Articles

Interaction that provides awareness and encouragement
Multimodal interaction
Knowledge processing andmemoryInteraction Control
Development
Execution
environment
environment

Recognition(Sensor data processing)Awareness andencouragement
Invoke conversation,
extract inner state andshow sympathy, andimprove understandability.

Communication based on
verbal and nonverbal media

Feedback(Actuator control)

Fig. 1. Cloud-based Interaction Control technologies for robotics integrated development environment (R-envTM).

4. Interaction Control technology
One of the key technologies necessary to realize theabove goals is the Interaction Control technology
used to connect humans and the devices that surroundthem. This technology will make it possible for the
robot to understand a situation by combining thecapabilities of various devices (e.g., voice from a
microphone, facial expressions from a camera, andhealth data from electronic health devices) and also todeliver messages that include emotion by combining
verbal and nonverbal communication methods. We
have developed this interaction technology as thecloud-based Interaction Control technology called‘R-envTM’ (Fig. 1).
In the past, a high level of background knowledgeof robot operating systems and programming languages was required in order to develop services
based on robots or applications that involved the collaboration and control of multiple devices. To spreadthe use of robot-enabled services, we believe it isnecessary to increase the number of application
developers and to communicate the requirements forknowledge-processing and robot control components
to facilitate the development of robot applications.
To do this, we built a new technology comprising arobot services execution environment and an integrated development environment (IDE). The former
environment is known as a PaaS (platform as a service), and it can permit a single system instance tosupport the use of multiple applications by multiple
3

tenants (users), enable management of mapping
between devices and users, and enable devices to beeasily connected. The latter environment involves an
IDE for software as a service (SaaS). The close cooperation between the execution environment and thedevelopment environment makes it possible for SaaS
to do the following:(1)	Simplify the debugging and execution of
applications(2)	Enable easy development of device feature
management(3)	Enable easy application development using
only a web browser up to a certain level
R-env enables anyone to easily add new devices toexisting services (Fig. 2). It is based on an architecture that supports advances in interaction technology
with the addition of the emotion-enhancing technology being developed at NTT [8].
The use of R-env in our homes will make it possible
to develop services that will integrate home appliances, healthcare devices, and robots with web services such as mail, schedulers, and weather forecasts.
Furthermore, extending the scope of R-env outside
the home will facilitate easy development of services
using personal devices such as smartphones (Fig. 3).
5. Use cases
An example of how the open innovation of R-env issupporting the robot vendor Vstone Co., Ltd. [9] isshown in Fig. 4. Vstone’s communication robot was
NTT Technical Review

Feature Articles

Technology for creating services by connecting multiple devices
Open innovation(Creating devices)

Scope of R-env(Creating services using devices)
Multiple intelligent devices

Motion control
Voice conversation

Intelligent devices(Service robots etc.)

Image recognition
Robot technology

Creating robots

IoT platform
Intelligent systems(Smart houses etc.)

Sensors
Big data
IoT technology

Creating novel services by integrating multiple
intelligent devices that could previously not be
achieved using a single device

Creating IoT systems

IoT: Internet of Things

Fig. 2. R-env for robotic device orchestration.

R-env
4 Well, if you insist,

why not?

1 It seems sunny outside.

6 Let me guide youTVthere!

2 You haven’t gone out

recently, have you?

IRo

bot

3 Why not go see theplace that was justshown on television?
Seems fun and relaxing!

Cleaning robot
5 We will clean thehouse while you aregone. Have fun!

Personal mobility agent

Fig. 3. Service image using R-env.

made intelligent by using NTT’s voice recognition
technology, dialogue control technology, speech synthesis technology, and sound-gathering technology.
We are conducting experiments to confirm that the
intelligent robot can, via R-env, collaborate with
peripheral devices and/or other robots. For example,
our intention is for robots to play an important role insupporting health management in places such as
Vol. 14 No. 5 May 2016

elderly housing and care facilities. The robots in thesefacilities will:(1)	Promote conversation and conduct healthmonitoring by talking with the elderly residents in collaboration with caregivers(2)	Provide personally tailored encouragement
based on the physical condition of each person

4Feature Articles

Service application

Voice recognition
Speech synthesis

UI for service development(web browser)

Dialogue control

Interaction engine

Services for the elderly
Watching over them and healthmanagement
Services for children
Public facilities

Cloud-robotics platform
Voice recognition

3D camera

Provided by Vstone

Robot device application

Provided by NTT DATA

Robot actuator
Robot device (SotaTM)

BLE devices

Provided by NTT MediaIntelligence Labs.

IO board

Provided by NTT Service
Evolution Labs.

Devices

BLE: Bluetooth Low Energy
IO: input and output
3D: three dimension
UI: user interface

Fig. 4. Formation of collaborative experiment between NTT DATA, NTT, and Vstone.

6. Conclusion
At NTT, by combining technology that will makedevices such as robots more intelligent with technology that will enable easy device collaboration, we
believe it is possible for robots to collaborate witheach other via the network and to support humanactivities and growth in various situations.

[4]
[5]

[6]
[7]

References
[1]
[2]
[3]

5

K. Čapek, “Robot,” Iwanami Shoten Publishers, 2003 (in Japanese).
M. Mizukawa, “Computers and Dynamics: NTT Human Interface
Laboratory Intelligent Robot Research Project,” Journal of Society ofBiomechanisms, Vol. 21, No. 1, pp. 43–45, 1997.
Press release issued by Sony Corporation in May 1999.
http://www.sony.net/SonyInfo/News/Press_Archive/199905/99-046/
index.html

[8]

[9]

Press release issued by Honda Motor Co., Ltd. in November 2000.
http://world.honda.com/news/2000/c001120.html
Y. Nakamura, S. Eitoku, Y. Iwata, M. Motegi, S. Mutoh, and M. Abe,
“Network Robot Platform Technology that Enables Cooperation of
Various Types of Robots,” NTT Technical Journal, Vol. 20, No. 1, pp.
22–27, 2008 (in Japanese).
http://www.ntt.co.jp/journal/0801/files/jn200801022.pdf
Press release issued by robot start inc. in September 2015 (in Japanese).
http://robotstart.co.jp/press10.pdf
MIC, “Population, Demographic Movement and Number of Households based on Basic Resident Register (as of March 31, 2013)” (inJapanese).
http://www.soumu.go.jp/main_content/000244523.pdf
T. Matsumoto, S. Seko, R. Aoki, A. Miyata, T. Watanabe, and T.
Yamada, “Affective Agents for Enhancing Emotional Experience,”
Proc. of HAI 2014 (the 2nd International Conference on HumanAgent Interaction), pp. 169–172, Tsukuba, Ibaraki, Japan, Oct. 2014.
NTT press release issued in July 2015 (in Japanese).
http://www.ntt.co.jp/news2015/1507/150728a.html

NTT Technical Review

Feature Articles

Tomohiro Yamada

Executive Research Engineer, Supervisor,
Networked Robot and Gadget Project, NTT Service Evolution Laboratories.
He received an M.S. in electronics engineeringfrom Niigata University in 1992 and an MBAfrom the University of Birmingham, UK, in2007. He joined NTT in 1992 and has been
engaged in research on content distribution systems and technologies to support human-robot
communication. He is the W3C (World Wide
Web Consortium) Advisory Committee Representative of NTT and also a member of the
Association for Computing Machinery, IEEE,and the Information Processing Society of Japan.

Vol. 14 No. 5 May 2016

Hiroshi Yoshikawa

Senior Research Engineer, Networked Robotand Gadget Project, NTT Service Evolution
Laboratories.
He received an M.S. in physics from Keio
University, Kanagawa, in 1995. He joined NTT
in 1995 and began conducting research on highdensity optical recording. He is now engaged inresearch on technologies that support andenhance human-robot communication.

6

Feature Articles: Artificial Intelligence Research
Activities in NTT GroupThe Eyes as an Indicator of the
Mind—A Key Element ofHeart-Touching-AI
Shigeto Furukawa, Makoto Yoneya, Hsin-I Liao,and Makio Kashino
Abstract

Technology for reading the mind is an important element of Heart-Touching-AI, which aims to understand and support the human mind and body. The well-known saying “The eyes are the windows to thesoul” expresses the fact that pupil response and eye movement can reveal information about a person’smental state. Research is underway at NTT Communication Science Laboratories to find the principles
of understanding a person’s mental state from his or her eyes in order to develop basic technologies that
can apply such a capability to real-world problems.
Keywords: eye movement, pupil, mind-reading

1. Introduction
Artificial intelligence (AI) continues to advanceand is being applied in numerous fields with various
objectives, including with the aim of achieving anenriching society that fulfills the needs of mind andbody. Accordingly, AI technologies have recently
gone beyond machine intelligence for making logicalinferences and are now being applied to understandhuman intellect, sensitivities, and emotions—all
aspects related to the human mind. Thanks to advances in machine learning and other technical fields,
there are now robots that can recognize human emotions by analyzing facial expressions and vocal patterns.
However, human emotions and psychological states
are not always expressed by such explicit information
as a person’s facial expressions or voice. A machine
would not surpass human performance in recognizing
another’s emotions as long as it uses only explicit
information. Consequently, by adding the ability tomeasure unconscious physiological responses suchas eye movement and heartbeat, NTT aims to achieve
AI that can read a person’s implicit mental state in a
1

way that only a computer could recognize and communicate with humans more effectively than evenhumans can.
The brain-computer interface (BCI) based on electroencephalographic measurements has progressed
as a basic technology for learning about a person’s
implicit mental state. This technology, however, hassignificant restrictions and suffers from high noiseand low robustness, and as a result, its use has largely
limited to certain fields such as medical care. Consequently, BCI is not considered to be ready for use ineveryday scenarios.
NTT Communication Science Laboratories hasdeveloped a basic technology for evaluating a person’s mental state by examining the person’s eyes, as
an alternative to electroencephalographic measurements. This technology has practical advantages inthat it enables eye-related information to be obtainedby a camera or other device in a non-invasive andnon-contact manner and enables data to be measured
without interfering with people’s natural movements
and sensations.

NTT Technical Review

Feature Articles

2. Information expressed by the eye
What kind of information can we obtain from thehuman eyes? The pupil, the black hole surrounded by
the iris, adjusts the amount of light admitted to theretina. The light reflex property of the pupil is wellknown; the pupil constricts in a bright environment(miosis) and dilates (mydriasis) in a dark environment. A number of studies have been published thatsuggest a relationship exists between pupil diameterand various cognitive processes such as target detection, perception, learning, memory, decision making(e.g., [1]). The factor that determines the pupil diameter is the balance between the mutually antagonist
actions of the pupil’s sphincter muscle and dilator
muscle, which are respectively governed by the parasympathetic nervous system and the sympathetic
nervous system. The pupil cannot be controlled consciously (voluntarily).
The eyeball, meanwhile, moves under the controlof the external eye muscle governed by the oculomotor nerve. Eye movement can be a voluntary process
as when turning one’s gaze toward something that
one wants to look at. However, it can also be involuntary in the case when one’s line of sight moves unconsciously toward a novel or conspicuous stimulus, orwhen small, jerk-like eye movements (microsaccades) occur in a state of visual fixation. Such involuntary eye movements have been said to reflect astate of caution induced by external stimuli or a stateof latent caution (not accompanied with a change ofgaze) [2]. Many evaluations of microsaccades havebeen done to measure their frequency of occurrence,
and we as well have taken up a detailed evaluation of
their dynamic characteristics and have been achieving results.
A variety of neural pathways contribute in a complex way to the control mechanisms governing pupil
diameter and eye movement. This system includes
nerve nuclei that are thought to have a deep relationship with cognitive processes. Nuclei whose functions are of particular interest include the locus coeruleus and the superior colliculus in the brain stem.
The locus coeruleus appears to play an importantrole in alertness, anxiety, stress, attention, and decision-making. These activities interact with various
regions of the brain via noradrenaline pathways, andthe pupils dilate through actions of the sympathetic
nervous system [1, 3].
The superior colliculus, meanwhile, has been said
to play a role in spatial attention and in controlling thedirection of sight, but recent research has revealed
Vol. 14 No. 5 May 2016that it can also interact with the pupils via the parasympathetic and sympathetic nervous systems [1, 2].
Neurons inside the superior colliculus react to stimuli associated with visual, somatic, and auditory sensations, and there is a spatial map in which the positionsof such neurons correspond to different points inspace where the stimuli appear. The superior colliculus not only receives input from lower-level sensory
systems that it transmits to higher-level sensory areasin the cortex, but it also receives input from thosehigher-level sensory and associated brain areas via
multiple pathways. It also receives projections fromthe basal ganglia and locus coeruleus as well. Reading the mind via pupil response and eye movement isbased on inferences of the cognitive processes thatare supported by this complex nervous system.
3. NTT activities in Heart-Touching-AI
We introduce here our recent efforts in developing
Heart-Touching-AI. A number of experiments haveconfirmed that the pupil and eye movements react tothe conspicuousness (saliency) of sound and to thedegree that the sound differs from what was expected(surprise). Some of these preliminary findings havebeen presented in previous articles in this journal[4, 5].
In the case of music, the saliency and surprise oftone and melody are thought to be important factorsthat determine a person’s affinity and preference for aparticular song or piece of music. We have created amodel for estimating a person’s affinity and preference for a piece of music by extracting appropriate
multidimensional features from pupil response andeye movement and using machine learning technology (Fig. 1). We found that with this model, it is possible to estimate a listener’s subjective music ratingscore with a certain level of accuracy [4].
A similar approach can be applied to evaluation of
stimuli other than music, for example, fondness (preference) for a face. In an experiment, we presented
images of faces to the observers while measuring
their pupil responses. At this time, miosis was foundto occur when the brightness of those images waschanged. However, it was also found that the extent of
miosis differed between the presented images eventhough the change in brightness was the same for all
images (Fig. 2). We asked the observers to subjectively evaluate facial-image preference and found thatthe extent of miosis corresponded to the observer’s
subjective evaluation [6]. Furthermore, using a model
similar to the one described above, we confirmed that2Feature Articles

Decode mental states from eye measurement.
Preference

Attention

Drowsinessor
Picture, music, …
Data
acquisition

Estimate(Classify)

Position (deg)Microsaccade
1
0.5
0
−0.5
−1
0

Classifier(Model)1

Time (s)2

Label(Evaluation)

Pupil size change(z-score)

Pupil size change
2

Modelconstruction

0
−3
0

1

2

Featureextraction

3

Time (s)Using novel method

Fig. 1. Model that estimates a person’s latent state of mind based on eyeball movements and pupillary reflex.

Attractiveness

100

Pupil diameter (arb. unit)

50
Low

0
−50
−100
−150

High

−200
−250
−0.5

0

0.5

1

1.5

Time after image onset (s)Fig. 2. Example of pupillary response to facial images.

an individual’s evaluation with respect to facial preferences could be predicted with high accuracy by
combining a variety of features including those of
pupil response and eye movement.
It would be natural to say that this correlation
between pupil diameter and facial-image preference3

obtained in the above experiments shows how thecognitive process associated with facial-image preference appears as a change in pupil diameter. However, we also consider the possibility of an inverse
cause-and-effect relationship, in which differences incertain features in individual images give rise to
NTT Technical Review

Feature Articles

differences in miosis, which in turn acts on the cognitive process.
To examine this possibility, we conducted an experiment on manipulating the amount of miosis by introducing a change in brightness to the backgroundpicture instead of to the facial images themselves [6].
We found that the subjective evaluation of facialimage preferences changed in conjunction with theamount of empirically introduced miosis. This finding has important implications. It can be said that thepupil not only expresses the state of mind but that it
is also a window that will enable AI to act on a person’s mind or mood.
4. Future outlook
In research up to now, the emphasis has been on
music preferences and facial preferences as targets of
estimation. However, targets that can be estimatedthrough eye measurements are not limited to preferences. As explained at the beginning of this article,
the pupil response and eye movement are governedby the complex nervous system. Therefore, it isinferred that the eyes latently reflect various cognitive
processes such as pleasure/displeasure, drowsiness,tension, and fear. The framework of feature extraction
and machine learning at the core of our technologies
is generic enough to be applied to any of those cognitive aspects (leaving aside the accuracy of the learning outcome).
However, a necessary condition here for themachine learning is that correct data are correctly
obtained. For example, to learn what types of eye
movements occur when people are sleepy, the eye
movements of a person in an actual state of drowsiness must be recorded. Up to now, it has been possible to obtain preferences related to targets such asmusic and facial images through questionnaires
administered in a laboratory, so it has been relatively
easy to obtain correct data. However, the estimationof feelings such as fear and anger is difficult to record
in a laboratory because of ethical issues, and it is alsodifficult to quantify subjective emotional states, evenby the observers themselves.
Another issue here is that pupil response and eye
movement can be affected by a variety of external andinternal factors, and it is therefore difficult to findsimple correspondence with specific cognitive processes or neural activity in real-world situations. In
future research, sophisticated psychological andbehavioral experiments and independent biologicalreaction measurements will be necessary to accuVol. 14 No. 5 May 2016

rately evaluate unconscious cognition. Given thatmany evaluation methods themselves have not yet
been established, the only way to solve these problems is to employ a variety of measurement andanalysis techniques while keeping their underlying
mechanisms and limitations in mind, thereby enhancing our understanding a little at a time. At NTT, we
are addressing these issues through basic researchthat spans diverse fields from physiology and psychophysics to machine learning technologies and throughmutual interaction among those fields.
Of course, development in measurement technology itself will also be necessary. The most common
approach to measuring pupil and eye movements is to
apply image processing to images captured by camera. Today, the smartphones, tablets, and personal
computers that we have all come to use are equippedwith web cameras, so we can consider that platforms
capable of measuring eye activity are expanding all
around us. The frame rate of a typical web camera,
however, is only several dozen frames per second(fps). If high-speed image capture becomes possible,
we can envision the catching of latent reactions ofshort duration such as the dynamic characteristics of
microsaccades. In our research, we are capturing
images at 1000 fps using specialized eye-measurement devices, so one key to applying this achievement to general devices will be the development ofhigh-speed cameras.
Provided that we accumulate basic knowledge, collect substantial data via sound experiments, andupgrade measurement technologies, we feel that the
day in which AI will be able to read a person’s mind
(or act on a person’s mind) by looking at that person’seyes is not that far off.
References
[1]
[2]
[3]

[4]

[5]

C. A. Wang and D. P. Munoz, “A Circuit for Pupil Orienting Responses: Implications for Cognitive Modulation of Pupil Size,” Current
Opinion in Neurobiology, Vol. 33, pp. 134–140, 2015.
S. Martinez-Conde, S. L. Macknik, X. G. Troncoso, and D. H. Hubel,“Microsaccades: a Neurophysiological Analysis,” Trends in Neuroscience, Vol. 32, No. 9, pp. 463–475, 2009.
G. Aston-Jones and J. D. Cohen, “An Integrative Theory of Locus
Coeruleus-norepinephrine Function: Adaptive Gain and Optimal Performance,” Annual Review of Neuroscience, Vol. 28, pp. 403–450,
2005.
M. Kashino, M. Yoneya, H.-I. Liao, and S. Furukawa, “Reading the
Implicit Mind from the Body,” NTT Technical Review, Vol. 12, No.
11, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201411fa6.html
S. Furukawa, S. Yamagishi, H.-I. Liao, M. Yoneya, S. Otsuka, and M.
Kashino, “Biological Measures that Reflect Auditory Perception,”
NTT Technical Review, Vol. 13, No. 11, 2015.

4

Feature Articles[6]

https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201511fa3.html
H.-I. Liao, S. Shimojo, and M. Kashino, “Correspondence between
Pupillary Response and Facial Attractiveness,” Oral presentation at
the 11th Asia-Pacific Conference on Vision, Singapore, 2015.

Shigeto Furukawa

Senior Research Scientist, Supervisor, Group
Leader of Sensory Resonance Research Group,
Human Information Science Laboratory, NTT
Communication Science Laboratories.
He received a B.E. and M.E. in environmental
and sanitary engineering from Kyoto University
in 1991 and 1993, and a Ph.D. in auditory perception from University of Cambridge, UK, in 1996.
He was a postdoctoral associate at Kresge Hearing Research Institute at the University of Michigan, USA, from 1996 to 2001, where he conducted electrophysiological studies on soundlocalization, specifically the representation ofauditory space in the auditory cortex. He joined
NTT Communication Science Laboratories in2001. Since then, he has been involved in studies
on auditory-space representation in the brainstem, assessing basic hearing functions, and thesalience of auditory objects or events. In addition, as the group leader of the Sensory Resonance Research Group, he is managing various
projects exploring mechanisms that underlie
explicit and implicit communication between
individuals. He was the principal investigator ofa Ministry of Internal Affairs and Communications SCOPE (Strategic Information and Communications R&D Promotion Program) commissioned research project on auditory salience. He
is a member of the Acoustical Society of America, the Acoustical Society of Japan (ASJ) (member of the Executive Council), the Association for
Research in Otolaryngology (ARO), and theJapan Neuroscience Society (JNSS).

Makoto Yoneya

Researcher, Sensory Resonance ResearchGroup, Human Information Science Laboratory,
NTT Communication Science Laboratories.
He received a B.E. and M.Sc. in engineeringfrom the University of Tokyo in 2010 and 2012.
He joined NTT Communication Science Laboratories in 2012 and has been studying biologicalsignal processing, especially of eye movements.
He is also interested in decoding people’s
thoughts based on brain or neural activity using
machine learning methods and has researched
decoding of the ‘internal voice’ using magnetoencephalography signals and multi-class SVM(support vector machine). He is also studying
auditory signal processing and is currently developing a mathematical model of auditory salience.
He received the 2011 Best Presentation Award
from the Vision Society of Japan. He is a memberof ASJ, JNSS, and ARO.

5

Hsin-I Liao

Senior Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
She received a B.S. and Ph.D. in psychology
from National Taiwan University in 2002 and2009. She joined NTT Communication Science
Laboratories in 2012 and has been studying auditory salience, music preference, and preferenceof visual images. She has also explored the use of
pupillary response recordings to correlate humancognitive functions such as auditory salience andpreference decisions. During 2007–2008, she
was a visiting student at California Institute ofTechnology, USA, where she studied visual preference using recorded eye movements and visualawareness using transcranial magnetic stimulation. She received a Best Student Poster Prize of
the Asia-Pacific Conference on Vision (APCV)in 2008, a Travel Award of the Association for the
Scientific Study of Consciousness (ASSC) in
2011, and a Registration Fee Exemption Awardof the International Multisensory Research
Forum (IMRF) in 2011. She is a member of the
Vision Sciences Society, ARO, and JNSS.

Makio Kashino

Senior Distinguished Scientist/Executive Manager of Human Information Science Laboratory,
NTT Communication Science Laboratories.
He received a B.A., M.A., and Ph.D. in psychophysics from the University of Tokyo in 1987,
1989, and 2000. He joined NTT in 1989. From1992 to 1993, he was a visiting scientist at the
University of Wisconsin (Prof. Richard Warren’slaboratory), USA. Currently, he is a visiting professor in the Department of Information Processing, Tokyo Institute of Technology (2006–), andPI (principal investigator) of a JST CREST project on implicit interpersonal information (2009–2015). He has been investigating functional andneural mechanisms of human cognition, especially auditory perception, cross-modal and sensorimotor interaction, and interpersonal communication through the use of psychophysicalexperiments, neuroimaging, physiologicalrecordings, and computational modeling.

NTT Technical Review

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Business Transformation Using
Artificial Intelligence at NTT
Communications
Koji Ito, Shoko Nishido, and Tomoaki Yamazaki
Abstract

At NTT Communications, we are actively introducing artificial intelligence (AI) in our operations,solutions, and services. In this article, we introduce some of the efforts underway to transform existing
business models using AI at NTT Communications.
Keywords: artificial intelligence, operation, IoT

1. Introduction

2. Enhanced services by using AI

NTT Communications held the NTT Communications Forum in October 2015 and had the highest
number of visitors ever. Exhibits related to Internet ofThings (IoT) and artificial intelligence (AI) attracted
a great deal of interest.
We are currently experiencing what has been
described as the arrival of the third AI boom. Leading
domestic and foreign information technology companies are intensifying their AI-related projects, and thework we are doing at NTT Communications has been
receiving a lot of interest from our customers in various fields.
There are three factors that have led to this AI
boom. The first is big data. With the advancement of
IoT, it is estimated that approximately 50 billion
devices will be connected to the network in 2020,
with even larger amounts of data collected from thosedevices. The second is the improvement in processing
capacity. We can now process complex calculations
that were not possible a decade ago. The third is thetechnological innovation of deep learning *1. This
technology has contributed to significant improvements in voice and image recognition accuracy. The
conditions are now such that this AI boom will not
simply end as a mere phase; we have the potential toapply AI to change the business structure itself invarious fields.

NTT Communications aims to improve its cloudservices based on its company slogan Global Cloud
Vision by using AI technology and also to contributeto its customers’ business innovations. We introduce
here some of the latest AI services and functions NTT
Communications plans to launch in 2016. These services embody the concept of Agent-AI that the NTT
Group is developing.

1

2.1 Cognitive Agent Service
The first one is the Cognitive Agent Service (Fig. 1).
This new service will understand and engage in natural language*2 interaction by using cutting-edge technologies including cognitive technology of IPsoft*3
and the advanced Japanese-language processing technologies that NTT Media Intelligence Laboratories
has been developing for four decades. This automatedservice will respond to a wide range of incoming*1 Deep learning: Multi-layer structured neural network that mimics
the structure of brain nerves; it can acquire a representation of
abstract data from fields such as image recognition and voice recognition and has the potential to greatly surpass conventional
technology.
*2 Natural language: Conversational and written language; language
people use in their everyday communication.
*3 IPsoft: Managed services and PaaS (platform as a service) provider; IPsoft’s business is expanding globally.

NTT Technical Review

Feature Articles

Direct, natural conversation

If not resolvable, transfer to operator.

Transfer

Asks questions

Answers andconfirms problems
End user
Confirm problem byasking questions.

Dictionary

Self-learning
Knowledge Manual for
responses& rulesCognitive Agent

Connects todatabase

Operator
Learn from operator’s
responses.

Automatically initiate
action (issue invoice etc.).

Fig. 1. Example solution: Cognitive Agent Service at contact center.

customer requests and help to diagnose and resolve
their problems.
This service will utilize not only basic morphological analysis but also dependency parsing, word conceptual vectorization, and question-answering technology that understands human intention in theirdialogues. These NTT technologies will enable the
new service to understand natural language and interact with customers smoothly. They are expected toresult in the service that differs from similar services
and provides a competitive advantage.
While the working population of Japan has been
declining, many companies have had to handle awider range of consumer demands at their contact
centers. To meet these demands, NTT Communications will launch this highly automated Cognitive
Agent Service in 2016.
2.2 AI technology for consumer services
NTT Communications is also planning to launch
new functions for its consumer services enhanced by
AI technology in 2016. The first function is AI labeling in its online storage service called My Pocket.
This will enable customers to categorize their photographs automatically into life events by using the AI
system. The second function will work on NTT Communications’ online household bookkeeping application called Kakeibon and will analyze its customers’
household accounts and offer financial advice.
Thus, NTT Communications expects to improve
the speed of upgrading its AI systems by self-learning
Vol. 14 No. 5 May 2016of knowledge and big data accumulated on its cloudat an accelerated pace. NTT Communications willdevelop its products to have a higher level of performance and quality with AI technology.
3. Use cases in customer service
For several years, we have collected and analyzed
big data related to service operations such as troubleshooting, service order and delivery records, inquiry
histories, and machine logs. However, we also haveexperienced the limitations of dealing with a hugeamount of data manually, and this is why we havefocused on AI.
We now introduce two use cases involving customer service.
3.1 Automatic forecasting of estimated time ofrepairWe have started to provide the estimated time ofrepair (ETR) in some of our network services. In thepast, we tried to calculate ETR based on each operator’s skill and experience. Therefore, the accuracy ofETR differed from operator to operator, and it turned
into another major task for us. Now, however, ETR iscalculated automatically by using big data and correlation analysis of trouble ticket data such as operation logs, alarms, and trouble patterns (Fig. 2). We arealso using AI to improve the accuracy of ETR with
self-learning. It continuously checks the pattern ofeach problem and the gap between the predicted ETR
2Feature Articles

Portal for operator

Operator

Calculates ETR

Master DB

Tickets

Checks ETR

Collecting data

Every 5 min.

Alarm

Decision tree

Comparing ETR withactual time of repair

Logs

Portal for user

Customer DB
Facility DB

ETR

Data
DB: database

Fig. 2. Automatic forecast of ETR.

value and the actual ETR value. Such data are accumulated and utilized by AI to improve accuracy. We
will strive to improve the accuracy of the forecasted
ETR and expand the application range to other services.
3.2 Improved customer service
Recently, in collaboration with NTT research anddevelopment laboratories, we have been testing ways
to improve customer service by utilizing AI in thefrequently asked question (FAQ) and question andanswer (Q&A) support sections. AI is being utilized
to search similar FAQs automatically and to give
feedback to improve its own precision (in conjunction with AI technology of Inbenta*4). It is also being
used to suggest a Q&A list that is related to the content of customer service calls to generate more appropriate answers in each case. Real-time speech-to-text,
FAQs applying intelligent search by natural language
processing, and automatic analysis of operation logs
are some example ways NTT Communications is
improving its customer service.
However, a sense of crisis that humans will be
replaced by AI has been increasing recently. On theother hand, by using AI, we can concentrate ourpower and resources more efficiently in places where
it is needed. Similarly, the advantages of using AI arealso attracting more attention than ever before. NTT
Communications will continuously work to improve3

customer satisfaction through the use of AI and to
apply its accumulated experience to support our business customers.
4. Use of deep learning to analyze the meaningof IoT data
Future IoT equipment*5 is not limited to consumers; it will become widespread in businesses from theindustrial field to the entire social infrastructure andis expected to continue growing rapidly. In suchcases, it is extremely difficult to analyze the wide
variety of big data by human power alone. In addition, each customer demands automated services
involving collecting and analyzing big data and taking actions based on them.
To address this situation, NTT Communications isfocusing on deep learning. In particular, we aim todevelop deep learning technology that can handle
time-series data and multi-modal data, because IoT
data is generated momentarily, and several different
types of data are complexly correlated with eachother (Fig. 3). This is Ambient-AI, another AI concept the NTT Group is working on.
*4 Inbenta: Natural language processing, AI applications, and FAQ
search solutions provider.
*5 IoT equipment: Equipment that can communicate with various
other devices that exist in the world.

NTT Technical Review

Feature Articles

Analysis with conventional two-dimensional deep-learning technology
Deep neural network
Result“Standing”?
“Bending over”?
Two-dimensional convolution
Analysis

Detected
resultAccuracy is uncertain because
analysis covers only one frame.

Analysis with three-dimensional time-series deep learning
Three-dimensional deep neural network
Time
Result“Squatting”

Three-dimensional convolution
Analysis

Detected
resultHigh-precision detection isachieved by analyzing theentire video with a time axis.

Fig. 3. Time-series deep learning.

In 2015, NTT Communications developed andevaluated time-series deep learning for analyzing themeaning of time-varying images (video). Normally,
when executing image recognition using deep learning, we input pixel values of the input image into aneural network for image recognition (typically usinga CNN (convolutional neural network), and the neural network repeatedly updates the weights between
layers based on the difference between the calculated
prediction results and the actual results. The neuralnetwork will continue in this manner in order toacquire an abstract feature to recognize the image. In
our work, we dealt with not only the target pixels of
the same frame as commonly used but also time information for treating successive frames at once. As aresult of this approach, the machine automaticallyunderstood what happened in the video with a highdegree of accuracy.
In consideration of possible real-world use casessuch as automatically monitoring a suspicious person
or a sick person, we tested this by identifying five
types of behavior: leaving an object, restlessness,squatting, standing, and walking, and we were able toVol. 14 No. 5 May 2016

identify each behavior using time-series deep learning with approximately 80% accuracy (Fig. 4).
In the near future, while still analyzing time-series
data, we also plan to investigate how to deal with avariety of data in a complex manner. Such data arereferred to as multi-modal data. To take advantage ofthis time-series deep learning, we are looking into
providing Video Analysis Platform Services (tentative name) that will enable the automated analysis ofvideo data in a variety of applications such as crime
prevention and marketing. This technology will makeit possible to detect suspicious behavior of people instores and manufacturing plants. It can also be
applied for crime prevention measures in a number offacilities that are expected to see an increase in thenumber of visitors in 2020, the year of the Tokyo
Olympic and Paralympic Games. We plan to workwith a variety of partners to improve this service andtechnology to a practical level.
Future work in AI, which is said to hold the key tothe fourth industrial revolution, has the potential tofundamentally alter the structure of various industries. We will continue to work on the challenge of4

Feature Articles(a) Identifying the behavior of “restlessness”(b) Identifying the behavior of “squatting”(c) Identifying the behavior of “leaving an
object”

Fig. 4. Image analysis using time-series deep learning.

creating new markets while utilizing AI technology inorder to become an ICT partner that contributes to thetransformation of existing businesses and the creationof a new business model.

Koji Ito

Senior Manager, AI Platform Strategic Unit,
Technology Department, NTT Communications
Corporation.
He received a B.E. and M.E. from Waseda
University, Tokyo, in 1996 and 1998. He joined
the Yokosuka Electrical Communications Laboratories of NTT in 1998 and studied user preference modeling using semantic web technology.
He joined NTT Communications in 2013. He iscurrently responsible for implementing an AI
platform that will automatically give added value
using deep neural networks, without any humanintervention (data scientists, specialists, etc.), tovarious kinds of data gathered in the cloud. He isa member of the Japanese Society for Artificial
Intelligence and the Institute of Electronics,
Information and Communication Engineers.

Tomoaki Yamazaki

Senior Manager, Innovation, Customer Services, NTT Communications Corporation.
He received a B.S. in physics from Tokyo University of Science in 1998. He joined NTT in1998 and contributed to developing various service systems involving digital video transmission
systems and communication systems (chat). He
is currently responsible for improving customer
satisfaction using big data analytics and AI.

Shoko Nishido

Senior Manager, Artificial Intelligent Taskforce, Applications and Content Department,
NTT Communications Corporation.
She received an LL.B. from Keio University,
Tokyo, and an MBA in international businessstrategy from Hitotsubashi University, Tokyo.
After joining NTT, she developed rate services
including Japan’s first flat rate service. Later, shejoined NTT Plala, formerly known as Plala Networks, and was in charge of content programming and production for the IPTV and video-ondemand service named Hikari-TV. She joined
NTT Communications in 2015 and is currently
responsible for developing new services based on
AI technology.

5

NTT Technical Review

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Artificial Intelligence Technology
Development and Its Practical Use
at NTT DATA
Otoya Shirotsuka
Abstract

NTT DATA established a new organization called the AI Solution Promotion Office on October 1,
2015, to promote artificial intelligence (AI)-related initiatives and to respond to customer inquiries
regarding the utilization of AI. The new organization will promote AI-related technological know-how
accumulated in-house, as well as the NTT research and development laboratories’ world-class AI technologies. This article introduces some examples of NTT DATA’s approaches regarding the utilization of
AI technologies.
Keywords: knowledge acquisition, communication robot, traffic simulation

1. Introduction
In recent years, NTT DATA has been receiving
inquiries from many customers asking how artificial
intelligence (AI) can be utilized in work operations
now that AI is experiencing exponential growth. The
majority of these inquiries are made with an underlying sense of crisis in which customers wonder if they
will be at a competitive disadvantage if they do not
introduce AI technology.
The AI Solution Promotion Office was establishedat NTT DATA on October 1, 2015 in order to respondto these kinds of customer inquiries regarding theutilization of AI and to promote AI-related initiatives.
One of the strengths of this division is utilizing NTT
research and development (R&D) laboratories’world-class AI technologies, but NTT DATA has also
been conducting its own business development andR&D of AI technologies since its establishment as acompany in 1988, and it excels at accumulating AIrelated technological know-how.
NTT DATA is focusing its AI utilization efforts inthree important domains: 1) middle and back office
operations such as examinations and analyses, 2)front office operations such as contact desk and callcenter support, and 3) various social infrastructures1

developed using large-scale data analysis. The firsttwo are Agent-AI target domains, and the third is an
Ambient-AI target domain [1]. Three specific examples of NTT DATA’s efforts regarding these domains
are described in this article: the efficient examinationof credit card member stores using knowledge acquisition technology, retail store customer support using
communication robots, and traffic congestion management using multi-agent simulations.
2. Efficient examination of credit card member
stores using knowledge acquisition technology
A major task in many industries involves examining the contents of applications, for example, credit
card applications, and carrying out the preliminary
work of gathering the necessary information todecide whether or not applications can be accepted.
Many of these operations are done manually and consist of repetitive tasks to confirm whether there are
any problems with the application contents. In addition, there are some cases in which specialized
knowledge is needed to make decisions on the contents. There is also a risk of losing business opportunities if too much time is consumed performing theexaminations. Consequently, these operations can
NTT Technical Review

Feature Articles

take a heavy toll on companies. It is for this reason
that NTT DATA is advancing R&D on examination
support technology that reduces the burden of these
operations.
2.1 R
 educed burden of credit card member store
management operations
The use of examination support technology began
two years ago, and results have already been obtainedon six cases.
The credit card industry is one area where results
have been obtained. This industry has seen an
increase in the number of scenarios in which consumers use their credit cards with the rapid growth of
e-commerce and the diversification of payment methods such as payment via smartphone. As a result,
there is strong demand to develop secure environments where customers feel at ease when using their
credit cards, and therefore, credit card companies areworking urgently to increase the security and efficiency of managing a wide variety of retail stores(member stores) that accept credit card payments.
As part of its efforts towards increasing the security
and efficiency of examination operations, NTT
DATA began working with Sumitomo Mitsui Card
Co., Ltd. in October 2013 to jointly initiate verification tests on the efficiency and automation of information gathering for member store examination, andin October 2014, these verification tests were implemented in Sumitomo Mitsui Card’s member store
management operations. Comprehensive decisions
are made after referring to and analyzing a variety of
information concerning member store examination
operations, but particularly in cases when examinations are conducted on non-face-to-face merchantssuch as e-commerce and smartphone stores.
For this reason, it is important to determine how
precisely and effectively this diverse information canbe analyzed within member store examination operations. Examination support technology extracts andcollects information necessary for examinations froma vast amount of information existing on the Internet
such as reputation information from forums, content
on e-commerce sites, and third-party information on
the Internet.
A dashboard function has also been implemented
that displays gathered information on one screen in
an easy-to-understand fashion and increases the efficiency of examination operations such as automatically highlighting and displaying areas that needattention. The AI Solution Promotion Office is focusing on R&D of knowledge acquisition technology
Vol. 14 No. 5 May 2016that specifies which areas are important in examination operations by adding meanings to text information. In the example of credit card member store
examinations, this technology is being utilized to not
only specify corporate names, addresses, and hoursof operation from e-commerce sites, but also to automatically gather the price range and price ceiling ofproducts that those corporations sell. In addition to
performing these tasks, this technology also gathers
information on the reputation of the relevant companies and checks to see if there are any entries onsocial networking sites regarding those companies.
2.2 S emi-automatic generation of rules throughthe use of machine learning
Knowledge acquisition technology is a technologythat estimates the structure of semi-structured andunstructured data such as sentences and lists. This is
a combination of original structuring technology inline with the data format of extracted targets. To utilize this technology, rules are necessary to extract
objective information, and to generate these rules,
there is a need for not only business knowledge but
also specialized knowledge on information extraction
technology*1, which is why specialists in the AI Solution Promotion Office have until now been receiving
requests and performing operations manually. Consequently, the considerable cost and time required forthis have been issues until now.
However, R&D is currently underway on technologies that semi-automatically create rules throughmachine learning by leveraging knowledge on rule
generation that has been conducted manually up until
now. For example, when creating rules that specify
place names from text information, a person first confirms whether the candidate place name (written in
red characters) on a screen listing example sentences
is correct or not by selecting either “o” or “x” (Fig. 1).
Next, the system uses machine learning based on thiscontent to automatically generate new rules. Repeating this step increases the accuracy of the rules. Thismechanism for generating rules and technology that
increases the efficiency of examinations is collectively called sensu (tentative name), and verification
tests on and application of sensu are currently progressing.

*1 Information extraction technology: One of several natural language processing technologies that extract useful informationfrom non-structured data such as text information, and converts
that information into a format that can be used by computers.

2

Feature Articles

When you want to automaticallygenerate rules that distinguishplace names

Select either “o” or “x” here.

Operator decides whether or not theword written in red characters is theplace name. If the word is the place
name, the operator selects “o,” if not
the operator selects “x.”For complicated tasks where supportcannot be given using automaticallygenerated rules, support is provided
by a specialist who understands
individual enrichment engines.

Fig. 1. Rules can be generated through machine learning by performing simple tasks.

3. Retail store customer support using
communication robots
The robotics market is expected to rapidly expand,
and there is an increasing expectation that the market
within the service field will grow quickly. However,issues still remain in terms of cost and technology; forexample, robots themselves are expensive, and thedata retained by robots must be protected. Accordingly, NTT DATA is making efforts to research anddevelop Cloud Robotics Platform*2 that simplifies therobots themselves and reduces their costs by gathering sophisticated functions on a cloud database andby integrating various devices such as robots and sensors connected to a network with various services on
the cloud.
3.1 Provision of advanced services at low cost
Since data collected from robots and sensor devices
are handled in an integrated manner, Cloud Robotics
Platform utilizes the R-envTM technology developed
by NTT Service Evolution Laboratories to control the
integration of cloud support devices as well as theknow-how accumulated by NTT DATA on M2M(machine to machine) and IoT (Internet of Things)(Fig. 2).
NTT DATA is already conducting verification tests
3

that utilize sensors and Cloud Robotics Platform.
Two examples are verification tests of elderly support
services in nursing homes and verification tests ofvisitor survey collected in public facilities. These
tests use Cloud Robotics Platform and also leveragetechnologies such as the communication robot
SotaTM*3 manufactured by Vstone Co., Ltd. and technology for speech recognition, dialogue control, andspeech synthesis developed by NTT Media Intelligence Laboratories.
3.2 S
 upport of guidance-related operations ofbank clerks using sensors and robots
The number of overseas visitors to Japan is expected to increase in 2020, and efforts are therefore
increasing to develop contact desk services utilizing
communication robots such as those that support customers in multiple languages at reception desks invarious industries. Accordingly, NTT DATA has been
paying attention to customer support-related operations in financial institutions where there are great
expectations for use scenarios involving communication*2 Cloud Robotics Platform: An information integration platformthat collects data from devices, analyzes the data, and controls
devices such as robots.
*3 “Sota” is a trademark of Vstone Co., Ltd.

NTT Technical Review

Services

Feature Articles

Examples of services utilizing robotics
Medical care &nursing

Reception &contact desks

Disasterprevention
measures

Touristinformation

EnergySysteminfrastructure

Cloud Robotics Platform
Situation recognition
Profileinformation

Object recognition

Robot
information

Spatial recognition

Location
information

Environment
information

Service robots

DevicesSensor devices (Internet of Things)

GPS

Spatialinformation

Control

Data gathering

Acceleration
sensorOperation
management

Biologicalinformation
sensorMotion detector

Sota™
Investigation& assistance

Reception &guidance

Communication

GPS: Global Positioning System

Fig. 2. Cloud Robotics Platform.

robots. NTT DATA and Vstone have been conducting
joint verification tests towards realizing customer
support using communication robots in actual
branches of financial institutions.
The feasibility of providing customer support services via dialogues with communication robots wasevaluated in verification tests conducted at the
RESONA Bank Toyosu Branch (Seven Days Plaza
Toyosu), a newly established branch, from the day of
opening on November 15, 2015, until the end of April
2016. During these verifications, we extracted issues
in business and operations while listening to the opinions of visitors and bank clerks. For example, highsensitivity sensors installed in the ceiling above thebuilding entrance on the first floor would detect people entering and leaving the facility and notify thecommunication robot stationed at the ATM (automated teller machine) corner. Along with communicating with customers by saying “Welcome!” or“Thank you for visiting us,” the robot directed visitors to the branch services located on the second
floor. High-sensitivity sensors installed at theentrance of the branch notified the communication
robot placed at the reception desk when visitors cameclose. The robot would then start a dialogue with thevisitor by greeting them with “Welcome!” and thenVol. 14 No. 5 May 2016

prompt the visitor to use the self-reception tablet
according to the contents of the robot’s conversation
with the visitor. This endeavor achieved services that
supported the guidance-related operations conventionally conducted by bank clerks (Fig. 3).
An increasing number of inquiries is being maderegarding the utilization of communication robots incustomer support-related operations. NTT DATA willcontinue conducting verification tests and is working
to expand the range of support that communication
robots provide in customer support-related operations such as customer guidance and product introductions in stores by utilizing AI technology. NTT
DATA also intends to promote the early practical useof communication robots.
4. Traffic congestion management using
multi-agent simulations
Traffic congestion is an important social issue that
affects both advanced and developing countries.
There are situations when applying basic solutions
such as expanding roads and improving intersections
is difficult due to cost and legal issues.
If the effects of multiple traffic congestion mitigation measures can be compared in traffic simulations,4Feature Articles

Visitor detection service
through the use of sensors

Customer support service
Communication
robot

Branch on second floor
Sensor

Self-receptiontabletGuidance
Bank clerk

Greeting

Cloud Robotics
Platform technology
Visitors

Customer
support

Sota™

Guidance tobranch on
second floor

Branch on
first floor

SensorSpeech recognition
Speech synthesis

Communication
robot

Dialogue control
Sensorintegration etc.
Notification

Greeting

OfficeSota™

Visitors

Fig. 3. Branch reception desk robot.

it will be possible to select the most effective measure. Accordingly, NTT DATA is conducting R&D on
technologies that predict traffic congestion and control stoplights by utilizing traffic simulation technology.

Center has developed graph data analysis processing
technology called equal granularity clustering (Fig.
5) with which NTT DATA has succeeded in conducting fast mesh partitions and conducting traffic simulation in real time.

4.1 L
 arge-scale traffic simulation technology
based on parallel distributed processing
NTT DATA’s traffic simulation adopts a method
using a multi-agent model. In this method, targets
that influence cars, stoplights, roads, intersections,and other traffic elements are regarded as agents, andthis method reproduces actual traffic congestion situations by operating an enormous number of theseagents on a virtual network of roads generated oncomputers. However, conventional measures require
high-spec computer resources to simulate situations
where a high density of cars causes congestion in a
broad area.
NTT DATA has successfully realized distributed
processing by separating an original map into a meshformat and by using multiple general-purpose servers(Fig. 4). In addition, since one mesh is assigned to
one core of a server and then calculated, distributed
processing requires mesh partitions with equal computational complexity. The NTT Software Innovation

4.2 M
 itigation of congestion by optimizing traffic
congestion in predictions and signal controls
To verify the effectiveness of traffic control using
this traffic simulation technology, NTT DATA cooperated with Jilin City in China in November andDecember 2014 in conducting verification tests in acity central area. During these tests, Jilin City buses
equipped with on-board computer terminals wereused to conduct traffic congestion prediction andstoplight control simulations in combination withstatistical information such as traffic volume, routes,
and information from approximately 200 probe units(the location and speed, etc., of each bus) concerning
the eight routes gathered from those on-board computer terminals. Traffic was controlled by reflecting
stoplight settings—which were optimized throughprior assessments done in simulations—in intersection stoplight machinery found within the city, andthe effects of traffic congestion mitigation and theimprovement of bus service times was verified. As a5

NTT Technical Review

Feature Articles

Map data
Separation intomesh format

Mesh

One mesh unit is assigned to one
simulator, and distributed processing
is performed.

Roads are partitioned
into areas thatcan be processedwith one CPU core.
Movement between meshunits is modeled by unifying
Connection between mesh units
the partitioned roads.
Mesh unit A

Road shared between
mesh units A and B
Mesh unit B

Capable of processing over 1 million probes
CPU: central processing unit

Fig. 4. Image of the use of distributed processing.

High-speed graph mining technology Grapon:
optimal mesh partition of road network in 0.1 milliseconds
Mesh partition

Mesh partition

Increasedtraffic

Fig. 5. Illustration of the application of equal granularity clustering technology.

result, the average speed of vehicles moving within
the target area was increased, traffic congestion was
reduced, and it was confirmed that the average bus
service time for target bus routes was improved on
average by 7% and up to 27%.
In 2015, NTT DATA cooperated with Guiyang Cityin China to develop technology that successfully
mitigates traffic congestion by performing traffic
congestion predictions and stoplight control simulations based on the data analysis results of cameras
installed in Guiyang City used for managing traffic.
This effort was done in collaboration with the InstiVol. 14 No. 5 May 2016

tute of Software, Chinese Academy of Sciences andwas conducted as a horizontal deployment of knowhow obtained from the verification tests conducted in
Jilin City.
Initiatives promoting the practical use of traffic
congestion mitigation solutions in combination withtraffic simulation and stoplight control technology
are progressing within NTT DATA in an effort
towards successfully managing major events in 2020
and realizing smart cities in countries around theworld.

6

Feature Articles

5. Future development
This article introduced three important domains
where AI technology is being utilized at NTT DATA,
related utilization examples, and various types of AI
technology applied in each of those cases. In order tobroaden the utilization of AI technology within theseimportant domains in the future, it will be essential
for AI technology itself to further progress. We at
NTT DATA will endeavor to strengthen our allianceswith NTT R&D laboratories and other business firms
as well as leading-edge research organizations suchas universities to develop an assortment of AI technologies with great potential in order to deal withsocial problems such as decreasing birthrates andaging populations, and to provide global support fordisaster prevention and other situations.
Reference[1]

T. Yamada, S. Takahashi, F. Naya, T. Ikebe, and S. Furukawa, “Artificial Intelligence Research Activities and Directions in the NTT
Group,” NTT Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa1.html

Otoya Shirotsuka

Executive R&D Specialist, Head of Promotion
Office for AI Solutions, Service Innovation Center, NTT DATA Corporation.
He received a B.A. in linguistics from the University of Tokyo in 1988 and joined NTT thesame year. In 1989, he moved to NTT DATA
Communications Systems Corporation (currently NTT DATA Corporation). From 1990 to 1991,
he was with SRI International as a visiting
researcher. His research interests include spoken
dialogue systems, text mining, and AI-based
decision support systems.

7

NTT Technical Review

Feature Articles: Artificial Intelligence Research
Activities in NTT Group

Utilization of Artificial Intelligence inCall Centers
Seiji Kawamura, Kenichi Machida, Kazuhira Matsui,
Daisuke Sakamoto, and Marie Ishii
Abstract

In November 2014, IBM Japan, Ltd. and Mizuho Bank, Ltd. issued a press release on the Watsoncomputer system that utilizes artificial intelligence (AI) to support call center operators. Since then, theexpectation that AI will be used in the call center sector of major companies has been increasing rapidly.
This article introduces AI technology that is being used effectively in call centers now and AI technology that is expected to be widely introduced within the next few years. NTT Software’s ForeSight Voice
Mining technology is also described.
Keywords: voice mining, artificial intelligence, call centers

1. Introduction
NTT is actively researching and developing four
artificial intelligence (AI) domains [1]. Of thosedomains, NTT Software is focusing its efforts on
Agent-AI. Specifically, NTT Software is developing
human-like agents that assist operators in call centers.
NTT Software launched a product named ForeSight
Voice Mining for call centers in May 2014. The product utilizes voice recognition technology, natural
language processing technology, and voice mining
technology developed by NTT Media Intelligence
Laboratories. Since releasing the product, NTT Software has been offering it mainly to call centers ofmajor financial institutions in cooperation with NTT
business companies.
In November 2014, IBM Japan, Ltd. and Mizuho
Bank, Ltd. issued a press release on the Watson system that uses AI to support call center operators [2,3]. Watson is defined as a cognitive computing system that understands/learns natural languages andsupports a human’s decision-making process. Since
this press release, there has been a growing awarenessof AI and of Watson in the call center sector of major
financial institutions, and consequently, the numberof inquiries to NTT Software asking about the capabilities of NTT’s AI technology has rapidly increased.
1

We introduce in this article the AI functions that ForeSight Voice Mining provides.
2. AI functions of ForeSight Voice Mining
ForeSight Voice Mining utilizes various AI element
technologies developed by NTT Media Intelligence
Laboratories such as machine learning and natural
language comprehension. It therefore provides functions that are useful in various business fields. These
functions include a response knowledge recommendation function to support operators’ work in callcenters and an automatic call summary function to
support the after-call work of operators. These functions are implemented with the image of virtual
agents (human-like agents). For example, call center
operators can see on their computer screen messages
from a virtual agent offering help or suggestions during their phone conversations with customers.
2.1 R esponse knowledge recommendation
function to support call center operators
Operators in call centers refer to various information and use know-how based on their experience tosolve the problems of customers when answering
calls. An issue that arises with less experienced
operators is that they tend to frequently place calls on
NTT Technical Review

Feature Articles

Various procedures are proposedbase on the customer saying, “I got
married last month.”Well, I got marriedlast month …
Tasks

Customer inf
info.
nffo.

Account
Account setting
Accou
sett
tting
ttin
t g

ReportsandRe
records

Monitoring System setting

Save

Back
List:

Attribute
Order: label_?
XXXX:
XXXX:

Name:
Post code:

Telephone
network“Congratulations.
Now, proceduresfor changing
address, name,beneficiary, etc. …”

Address:

Call

Phone no.:

XXXX:
XXXX:
XXXX:

Current
item
Input the current contract of the customer.

Apply

XXXXXXXXXX:XXXXX:XXXXXXX:
Order item
Input items ordered by the customer.
XXXX:
Sales
representative

Input the name of sales representative.

Fig. 1. Response knowledge recommendation function.

hold or to prolong the response time in order to referto such information, and this makes customers impatient and increases the operating cost of call centers.
As a solution to these problems, ForeSight Voice
Mining has a function in which a virtual agent listens
to the conversation between operators and customers
and promptly gives suggestions on appropriate
responses to the operator. This function uses voicerecognition of calls on a real-time basis and natural
language processing technology to display appropriate responses on the operators’ computer screens, andit enables operators to automatically obtain appropriate response knowledge during their conversations
with customers and to respond to calls more quickly
and effectively.
For example, at a call center of a life insurancecompany, when a customer says, “I got married lastmonth,” a virtual agent responds to the word married
and instantly displays relevant information on thescreen of operators such as Procedures for addresschange, Procedures for name change, and Procedures for beneficiary change. In this way, sharing theknow-how of experienced operators with the entire
call center staff enables less experienced operators towork as effectively with customers as experienced
operators, improves customer satisfaction, and reduces the operating cost of the call center (Fig. 1).
Vol. 14 No. 5 May 2016

2.2 Automatic call summary function to support
after-call work of operators
Call center operators enter the content of their conversations with customers in the system as response
records after completing calls. This is referred to as
after-call work. The difficulty with this work is thatthe description of response records entered as aftercall work varies widely depending on operators, andthe length of descriptions significantly decreases during busy times.
ForeSight Voice Mining has an automatic call summary function in which a virtual agent summarizes
the conversation between operators and customers
and proposes summarized sentences to the operator.
This function realizes role sharing between machinesand humans; that is, AI (machine) is responsible forpreventing variations in the lengths of descriptions in
response records between busy periods and slow periods and variations in viewpoints of descriptions bydifferent operators, and operators (humans) areresponsible for handling calls, which requires them torespond appropriately to customers at all times,
whether the customer is exhibiting a pleasant orunpleasant manner. In other words, AI is utilized to
substitute for and support a certain part of humanactivities, and AI and humans take charge of theirown areas of specialty.
2

Feature Articles

Summaries created by the automatic call summaryfunction can be categorized into two groups; one is
automatic extraction of the nature of the call to
extract the reasons customers called in the first place,and the other is automatic extraction of importantphrases to extract only important remarks during thecall. In this method, summaries created do not dependon the duration of the call and the key points of calls
to be recorded in a concise way. Another method
called the unnecessary sentence deletion method canalso be used for the summary function. The function
creates summaries by deleting unnecessary sentences(such as greetings and supportive responses) included
in conversations. This method has a low risk of missing information to be recorded and is able to record
an overview of the entire call. However, with thismethod, the length of summaries depends on the
duration of the call. Because the summary function
method to be used is determined according to how thecompanies will use the summaries later, NTT Software proposes the optimal method for customers
depending on their intended use of summaries.
The use of the response knowledge recommendation function and the automatic call summary function described above requires prior learning exercises
by using recorded voices of actual incoming calls tocall centers as well as response records before applying the functions in actual situations. Prior learning
exercises enable the functions to be adjusted to thework of companies where the functions are to be
introduced, and regular additional learning after thestart of operation improves the ability of the functions
to adapt to the work.
3. Technological advances leading to increased
interest in AI use in call centers
There was a boom in interactive voice response(IVR) systems based on voice recognition among callcenters from the late 1990s to the early 2000s. In
Japan, the U.S. firm Nuance Communications, Inc.
and other vendors of voice recognition systems promoted the introduction of the IVR system using voicerecognition in some innovative companies. Most
voice recognition systems introduced at that time
were not able to recognize conversations. They only
recognized words (or a series of words) registered in
the system beforehand. Now, about 20 years havepassed since then, and the throughput capacity ofcomputers has dramatically improved, deep neuralnetworks have been introduced in the algorithms ofvoice recognition engines, and sufficient accuracy of
3voice recognition has been achieved for conversations speakers (operators and customers) have without even being aware of voice recognition [4]. Theseadvances in technology prompted IBM Japan andMizuho Bank to issue the press release about Watsonin November 2014, which led to the rapid increase in
expectations for the use of AI in call centers.
4. Improved corporate image through the use of
AI in call centers
In many cases, call centers of companies represent
a sector that is costly, and therefore, reducing costs atcall centers is a never-ending task. At the same time,
however, improving customer satisfaction and maintaining/improving corporate brands require better
customer support, and a high-quality customer experience at call centers is an important factor to maintain/improve brands. In other words, for companies
that operate call centers, the coexistence of cost
reduction and a high-quality customer experience is aconsistent objective. Innovative companies are willing to introduce AI (machine) that will take charge of
dealing with customers to reduce costs, and they seek
to convey the image of being cutting-edge companies
that adopt advanced technology quickly. Conventional thinking used to be that a high-quality customer experience could only be achieved whenhuman operators interact with customers. However,this idea is changing, and it is more often thought that
improvement of voice recognition and natural language processing technologies will make it possible
to achieve a high-quality customer experience inwhich AI interacts with customers in some areas to asatisfactory extent. Using AI in customer service will
make it possible to achieve stable quality in dealing
with customers without having to deal with difficulties in securing sufficient human resources. In addition, a company’s corporate image will greatly
improve if they can impress customers with the highquality responses of their AI systems. Therefore,
innovative companies are now starting to compete intheir use of AI to provide customer support.
5. Future development
NTT Software is using the advanced technology of
NTT Media Intelligence Laboratories and promoting
products and services in which AI directly interacts
with customers. For example, NTT Software is pursing virtual agents that understand and properly
respond to what customers say when answering calls
NTT Technical Review

Feature Articles

and virtual agents that understand and properly
respond to sentences input into websites and social
networking services by customers. NTT Software
believes that such products and services cannot be
achieved overnight but can be achieved within a few
years, as strong needs for them exist, and technologies to meet those needs are steadily advancing.

References
[1]

[2]
[3]
[4]

Seiji Kawamura

T. Yamada, S. Takahashi, F. Naya, T. Ikebe, and S. Furukawa, “Artificial Intelligence Research Activities and Directions in the NTT
Group,” NTT Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa1.html
Press release issued by IBM Japan on November 6, 2014 (in Japanese).
http://www-03.ibm.com/press/jp/ja/pressrelease/48712.wss
Press release issued by Mizuho Bank on November 6, 2014 (in Japanese).
http://www.mizuhobank.co.jp/release/2014/pdf/news141106.pdf
Yano Research Institute, “Potential Market Reform by AI and Big
Data 2015,” Oct. 2015 (in Japanese).

Daisuke Sakamoto

Executive Project Manager, Multimedia andMobile Business Department, NTT Software
Corporation.
He joined NTT Software in 1989 and is developing natural language processing application
systems and voice recognition and synthesis
application systems.

Engineer, Multimedia and Mobile Business
Department, NTT Software Corporation.
He joined NTT Software in 2011 and is working on the development of the ForeSight Voice
Mining system.

Kenichi MachidaMarie Ishii

Senior Engineer, Multimedia and Mobile Business Department, NTT Software Corporation.
He joined NTT Software in 1998 and was
active in developing network service application
systems. He has been researching voice recognition and synthesis application systems since
2009.

Engineer, Multimedia and Mobile Business
Department, NTT Software Corporation.
She joined NTT Software in 2011 and developed software test tools. She has been working asa system engineer of the ForeSight Voice Mining
system since 2013.

Kazuhira Matsui

Chief Engineer, Multimedia and Mobile Business Department, NTT Software Corporation.
He joined NTT Software in 2001 and developed electronic commerce systems. He is now
leading the development of the ForeSight Voice
Mining system.

Vol. 14 No. 5 May 20164

Regular Articles

Optical Switches Using Beam
Steering by Computer Generated
Hologram
Keita Yamaguchi, Kenya Suzuki, and Joji Yamaguchi
Abstract

We describe a computer generated hologram (CGH) method that is applicable to a multiple input andmultiple output (MxN) optical switch based on liquid crystal on silicon (LCOS). The optics of theconventional MxN optical switch require multiple spatial light modulations. In addition, a phase patterndesigned using the CGH method achieves a simple MxN optical switch with a single spatial phasemodulation. Moreover, the intrinsic loss of the proposed MxN switch from beam splitting can be reducedby routing multiple signals with a single knob control. We demonstrate a 4x4 wavelength selective
switch and a 2-degree reconfigurable optical add/drop multiplexer switch based on the above CGH
method. The experimental results indicate that these switches work well with a crosstalk of < –20.0 dB.
Keywords: hologram, optical switch, LCOS

1. Introduction
In optical communication networks, wavelength
selective switches (WSSs) [1, 2] are used to achieve
wavelength provisioning. WSSs can route a specificwavelength channel in a wavelength division multiplexing (WDM) system in any direction without
interrupting the remaining channels. WSSs with freespace optics are widely used in networks in order toachieve a high port count and low crosstalk. An optical system of a typical WSS with one input and multiple output ports (1xN) is shown in Fig. 1 [1, 2].
Liquid crystal on silicon (LCOS) is used for theswitching engine. The WDM signals from the input
port are demultiplexed by the grating, and each wavelength channel is focused at a different position on the
LCOS. The LCOS modulates the wavefront of an
input beam at each wavelength channel independently by using pixels arranged in two dimensions.
After the wavefront is modulated, the beams are multiplexed by the grating and then connected to one ofthe output ports. The spatial phase pattern for themodulation controls the diffraction angles of input
beams. Therefore, the focus point on a line with out1

put ports can be controlled by such patterns, leading
to port switching. In these optics, multiple signals
with the same wavelength input from other ports arefocused at the same position with different incident
angles and modulated by the same phase pattern on
the LCOS. Therefore, a WSS cannot control multiple
incoming signals with the same wavelength independently.
Multiple input and multiple output (MxN) opticalswitches have been studied recently as a way toachieve flexible switching [3, 4]. The MxN switch
can route multiple signals from multiple directions
with one module. For example, a multi-degree reconfigurable optical add/drop multiplexer (ROADM),
which needs the MxN switching function, can berealized by using multiple 1xN WSSs and splitters[5]. A 4-degree ROADM node with a broadcast-andselect (B&S) configuration is shown in Fig. 2. Thisconfiguration requires four 1xN WSSs and four splitters, which is costly and requires a lot of space. The
conventional approach to the MxN optical switch is amultiple beam-steering configuration. However, theoptics of such a switch are more complex than thoseof the 1xN WSS, which increases the size and cost. In
NTT Technical Review

Regular Articles

Dispersion
planex

Input/output
portz

Dispersion
Switching Dispersion
LCOS
lens
lens
lens
Grating

Switching
plane
y
z

Fig. 1. Optical system of 1xN WSS.

LCOS

In-4
In-3
In-2
In-1
WSS

Holographic pattern

WSS

Beam split

WSS

Out-1

WSS
Splitter

Out-2
Diffraction angle control Lens

Phase

Fig. 2. Schematic of 4-degree ROADM node with B&S
configuration.

Fig. 3. O ptical configuration of 4x2 optical switch based on
CGH.

this study, we added the MxN switching function to
the 2-f optics by using a hologram method.

phase pattern on the LCOS was designed by using thecomputer generated hologram (CGH) method. In thecalculation of a CGH, the phase pattern is randomly
modified along the switching axis. Transmittances
for designated ports are then calculated in simulations. These steps are iterated until the transmittances
for the ports achieve a target value.

2. MxN optical switch using holographic
phase modulation
Our MxN switch integrates the B&S function byusing holographic phase modulation. The B&S function can be divided into beam splitting (broadcast
function) and port selecting (select function). In
LCOS 2-f optics, the broadcast function refers to thediffraction of an input beam into multiple angles. The
select function refers to the control of the diffraction
angle by the phase modulation on the LCOS. If these
two functions can be integrated, a B&S type MxN
switch can be achieved with the 2-f optics. Holographic phase modulation can achieve both of thesefunctions with 2-f optics. In this study, the spatialVol. 14 No. 5 May 2016

2.1 B&S type MxN switch
An MxN switch that integrates the B&S function
via one-time holographic phase modulation has previously been proposed [6, 7]. A 4x2 switch is shownin Fig. 3. This switch has the same 2-f optics as the
1xN switch. In the optics, all input beams are split
and diffracted at multiple angles (broadcast function).
The signal beam connected to each output port isselected from each split bundle of beams by adjusting
the diffraction angle (select function). The phase2

Regular Articles

Lens

2-pole 2-throwswitch

LCOS

In-1
In-2
Out-1A

In-2

Out-2A
Out-1B

In-1
Pole

Out-2B
Throw(a)Out-1A
Out-1B
Out-2A
Out-2B(b)

Phase

Fig. 4. (a) Switching function and (b) optical configuration of 2-pole 2-throw switch.

pattern, designed by using the CGH method on the
LCOS, can split input beams into multiple angles andindependently control the diffraction angle of eachbundle of split beams [8]. Moreover, the connection
state of multiple signals can be switched by thechange in the phase pattern. As a result, an input signal from any input port can connect to any output port
by using the CGH pattern.
This switch has intrinsic loss caused by beam splitting, which increases with the number of input
beams. When the simultaneous switching of multiple
signals is enabled, the multi-pole multi-throw(MPMT) switching function can reduce the ramification number and the intrinsic loss in the MxN switch[9].
2.2 MPMT switch array
The MPMT switch controls multiple signals in a set[10]. The function of a 2-pole 2-throw switch is illustrated in Fig. 4(a). In-1 and in-2 can be connected toout-1A or out-1B and out-2A or out-2B, respectively.
This switch can control the connection states of two
input signals from in-1 and in-2 with a single knob
control. Although the switch controls MxN ports, theintrinsic loss caused by beam splitting does not occur.
The optical system of the 2-pole 2-throw switch with
2-f LCOS optics is shown in Fig. 4(b). Here, the difference in the diffraction angles from the LCOS is thesame as the difference in the incident angles. Therefore, if the output angle differences between the output ports are the same as those of the incident angles
between the input beams, multiple input beams frommultiple input ports connect to multiple output ports
simultaneously by single spatial phase modulation.
This is the MPMT function.
When multiple input beams are always switched in3

a set, the intrinsic loss from beam splitting can be
reduced by using the MPMT function. For example,
in ROADM networks, the same wavelength channel
in the WDM system is used by received and transmitted signals connected to the same node. In otherwords, when an in-port and drop-port are connected,
an input signal from an add-port, which is connectedin the same direction as the in-port, should be connected to an out-port. In this case, these input signals
from the in-port and add-port can be controlled byusing the MPMT function.
The switching function of a 2-degree ROADM
switch based on the 4x4 switch with MPMT function
is shown in Fig. 5(a). There are two connection
states, that is, the add/drop state and the through state.
In the add/drop state, west-in and east-in are respectively connected to the west-drop and east-drop ports.
At this time, west-add and east-add are respectively
connected to west-out and east-out. In the throughstate, west-in and east-in are respectively connectedto east-out and west-out. The others are not connected
to any ports. Here, the input signals in bundle-1 andbundle-2 can be controlled independently. This 4x4
switch with MPMT function can reduce the ramification number in the 4x4 switch from four to two compared with the B&S type 4x4 switch, which can
independently route all input beams, resulting in a
reduction of the intrinsic loss from 6 dB to 3 dB. An
optical configuration of the proposed multicastMPMT switch is shown in Fig. 5(b). The configuration is the same as that of the CGH based MxN
switch. We arrange the output ports so that the difference between diffraction angles equals the difference
between diffraction angles connecting the out ports,
leading to the MPMT function.

NTT Technical Review

Regular Articles

Bundle-2

Lens

MPMT switch
SplitterWest-inEast-addWest-add
East-in

LCOS

West-inEast-addWest-add
East-inWest-out

Holographic pattern

East-drop
East-out
West-drop
Bundle-1

West-drop
Bundle-1

East-dropWest-out

East-out

Bundle-2
(a)

Phase
(b)

Fig. 5. (a) Switching function and (b) schematic illustration of 2-degree ROADM switch based on 4x4 switch with
MPMT function.

In-1

In-2

In-3

In-4

Out-1

T (dB)

−15
−25
−35
−45

Out-2

T (dB)

−15
−25
−35
−45

Out-3

T (dB)

−15
−25
−35
−45

Out-4

T (dB)

−15
−25
−35
−45
1542

λ1

λ2 λ3

1544
1546
Wavelength (nm)

1548

1542

1544
1546
Wavelength (nm)1548 1542

1544
1546
Wavelength (nm)1548 1542

1544
1546
Wavelength (nm)1548

Fig. 6. Transmission spectra of B&S type 4x4 WSS based on CGH.

3. Experimental results
We describe here the experimental results for the
B&S type 4x4 WSS, which can connect any input andany output ports, and the 2-degree ROADM switch
with MPMT function. The patterns for spatial phasemodulation on the LCOS were designed by using the
CGH method [11]. The wavelength band was divided
into three segments in the 4x4 WSS and two segmentsin the 2-degree ROADM switch. These separated
wavelength bands were indexed as λ1, λ2, and λ3 fromshort to long wavelength.
Vol. 14 No. 5 May 2016

The experimental results are shown in Fig. 6. Diagrams lying in the same horizontal line show themeasured transmittance (T) for the same output port,
and diagrams lying in the same vertical line show themeasured T from the same input port. Here, eachwavelength channel is set in each switching state, as
indicated in Table 1. This result shows that this 4x4
switch worked well. At this time, the maximum port
crosstalk (XT) was –20.0 dB, and the increase in
insertion loss above the basic level for switching was
6 dB for beam splitting and from 0.9 dB to 3.1 dB due
to imperfections in the CGH.
4

Regular Articles

Table 1. Connection states in experiment performed on B&S type 4x4 WSS based on CGH.
λ1

In-1

In-2

In-3

In-4

Out-4

Out-1

Out-2

Out-3

λ2

Out-4

Out-3

Out-2

Out-1

λ3

Out-1

Out-2

Out-3

Out-4

East-out

West-in
λ1

West-out

λ2

East-inWest West East
-add -drop -add

East
-drop

(a)
East-inWest-inWest-add

λ1

East
-out

λ2

T (dB)T (dB)

−35

−25

West
-drop

−35
−45
1542

−25
−35
−45
−15

1544
Wavelength (nm)1546 1542

1544

1546

T (dB)

East
-drop

−25

−45
−15

T (dB)

Westout

East-add

−15

−15

−25
−35
−45
1542

Wavelength (nm)15441546 1542

Wavelength (nm)

1544

1546

Wavelength (nm)(b)

Fig. 7. (a) Connection plan and (b) transmission spectra of proposed 2-degree ROADM switch with MPMT function.

The experimental results with the 2-degree ROADM
node with MPMT function are shown in Figs. 7(a)and (b). Channels λ1 and λ2 were set in the add/drop
and through states, respectively, as shown in Fig. 7(a).
The increase in the insertion loss from the MPMT
switching function was 3 dB for beam splitting and
3 dB to 4.1 dB due to the imperfection of the CGH.
The maximum port XT was less than –20.0 dB.
4. Conclusion
We reported on MxN optical switches with 2-f
optics. These switches use holographic spatial phasemodulation for the integration of the B&S function.
The switches integrate the B&S function, which cancontrol multiple input signals independently, byusing a phase modulation pattern designed using the
CGH method. The CGH pattern can split input beams(broadcast-function) and control the diffraction angleof split beams (select-function). Moreover, the
MPMT function can reduce the intrinsic loss of thisMxN switch. We found that in a B&S type 4x4 switch
5

without the MPMT function, the increase in insertion
loss above the level for basic switching was 6 dB for
beam splitting and from 0.9 dB to 3.1 dB due toimperfections in the hologram. In comparison, in a
4x4 switch with the MPMT function, the increase in
insertion loss above the level for basic switching was
reduced from 6 dB to 3 dB for beam splitting andfrom 3 dB to 4.1 dB due to imperfections in the hologram. Moreover, the crosstalk between ports was lessthan –20 dB.
The port XT and insertion loss can be improved by
optimizing the design of the phase modulation patterns. We expect that the development of this switch
will lead to new applications.
References
[1]

[2]

X. Gu, K. Seno, H. Tanobe, and F. Koyama, “60-channel Wavelength
Selective Switch on Bragg Reflector Waveguides Array with 125
Output-ports,” Proc. of ECOC (European Conference on OpticalCommunication), We.3.5.3, Cannes, France, 2014.
K. Suzuki, Y. Ikuma, E. Hashimoto, K. Yamaguchi, M. Itoh, and T.
Takahashi, “Ultra-high Port Count Wavelength Selective Switch
Employing Waveguide-based I/O Frontend,” Proc. of the Optical

NTT Technical Review

Regular Articles

[3]
[4]

[5]
[6]

[7]

Fiber Communications Conference and Exhibition (OFC), Tu3A.7,
Los Angeles, CA, USA, 2015.
N. K. Fontaine, R. Ryf, and D. T. Neilson, “N×M Wavelength
Selective Crossconnect with Flexible Passbands,” Proc. of OFC,PDP5B.2, Los Angeles, CA, USA, 2012.
H. Uetsuka, M. Tachikura, H. Kawashima, K. Sorimoto, H. Tsuda, K.
Sasaki, and Y. Yamashita, “5×5 Wavelength Cross-connect Switch
with Densely Integrated MEMS Mirrors,” Proc. of Photonics in
Switching, PW2B.2, San Diego, CA, USA, 2014.
P. Roorda and B. Collings, “Evolution to Colorless and Directionless
ROADM Architectures,” Proc. of OFC, NWE2, San Diego, CA, USA,2008.
J. B. Schroeder, J. A. Carpenter, S. Frisken, M. A. Roelens, and B. J.
Eggleton, “6 Port 3x3 Wavelength Selective Cross-connect by
Software-only Reprogramming of a 1xN Wavelength Selective
Switch,” Proc. of OFC, W2A.15, Los Angeles, CA, USA, 2015.
K. Yamaguchi, M. Nakajima, Y. Ikuma, K. Suzuki, J. Yamaguchi, andT. Hashimoto, “MxN Wavelength Selective Switches Using Beam

Keita Yamaguchi

Researcher, Optoelectronics Integration
Research Group, Photonics-Electronics Convergence Laboratory, NTT Device Technology
Laboratories.
He received a B.S. and M.S. in physics from
Tsukuba University in 2009 and 2011. He joined
NTT Microsystem Integration Laboratories in
2011, where he conducted research on WSS
systems. He has recently been researching
LCOS-based WSS and holographic phase modulation. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE).

Splitting by Space Light Modulators,” Proc. of the Opto-Electronics
and Communications Conference, PWe.08, Shanghai, China, 2015.
[8] H. Yang, B. Robertson, and D. Chu, “Crosstalk Reduction in
Holographic Wavelength Selective Switches Based on Phase-only
LCOS Devices,” Proc. of OFC, Th2A.23, San Francisco, CA, USA,2014.
[9] K. Yamaguchi, M. Nakajima, Y. Ikuma, K. Suzuki, J. Yamaguchi, M.
Itoh, and T. Hashimoto, “Optical Multi-cast Multi-pole Multi-throw
Switch Using Holography and Its Application to 2-degree ROADM
Node,” Proc. of Photonics in Switching, WeII3.3, Florence, Italy,
2015.
[10] P. Mekanand, D. Puttadilok, and D. Eungdamrong, “Double Pole Four
Throw CMOS Switch in a Transceiver of MIMO Systems,” Proc. of
the International Conference on Advanced Communication
Technology, pp. 472–474, Phoenix Park, Korea, 2009.
[11] Y. Maeda, H. Hirano, and Y. Kanata, “A Learning Rule of Neural
Networks via Simultaneous Perturbation and Its Hardware
Implementation,” Neural Networks, Vol. 8, No. 2, pp. 251–259, 1995.

Joji Yamaguchi

Senior Research Engineer, Supervisor, NTT
Device Innovation Center.
He received a B.E., M.E., and Ph.D. inmechanical engineering from the Tokyo Instituteof Technology in 1988, 1990, and 1993. In 1993,
he joined NTT Interdisciplinary Research Laboratories, where he engaged in research on opticalcross-connect systems. During 2000–2001, hestudied microelectromechanical system control
technology as a visiting researcher at the University of California, Berkeley, CA, USA. He is amember of the Japan Society of MechanicalEngineers and The Japan Society for Precision
Engineering.

Kenya Suzuki

Senior Research Engineer, Supervisor, Optoelectronics Integration Research Group, Photonics-Electronics Convergence Laboratory, NTT
Device Technology Laboratories.
He received a B.E. and M.E. in electrical engineering and a Dr. Eng. in electronic engineeringfrom the University of Tokyo in 1995, 1997, and2000. He joined NTT in 2000. From September
2004 to September 2005, he was a visiting scientist at the Research Laboratory of Electronics(RLE) at the Massachusetts Institute of Technology. From 2008 to 2010, he was with NTT
Electronics Corporation, where he worked on thedevelopment and commercialization of silicabased waveguide devices. He has also been aguest chair professor at the Tokyo Institute ofTechnology since 2014. His research interests
include optical circuit design and optical signal
processing. He received the Young Engineer
Award from IEICE in 2003. He is a member of
IEICE, the Institute of Electrical and Electronics
Engineers, and The Physical Society of Japan.

Vol. 14 No. 5 May 20166

Regular Articles

Edge Router System that Distributes
Traffic Flexibly According to Services
Masaaki Omotani, Takeshi Osaka, Ichiro Kudo,
Akiko Kuboniwa, Chiharu Morioka, Taizo Yamamoto,Yuta Watanabe, Akira Misawa, and Tsukasa Okamoto
Abstract

In the future, communication networks will need to provide various services more flexibly andefficiently according to the diversified needs of customers and service partners. NTT Network Service
Systems Laboratories is researching and developing an edge router system that distributes traffic flexibly
to various service functions in a network. This system is being developed in order to reduce costs byincreasing transfer capacity and reducing power consumption, and to respond to service needs flexibly
by employing traffic distribution and policy control functions.
Keywords: edge router system, DPI, traffic control

1. Introduction
The Internet of Things (IoT) is a network of various
physical objects—things that exchange data and have
network connectivity. As IoT development has progressed and 4K/8K video has become popular, thedemand for information communications has been
increasing. The required quality and capabilities ofcommunications vary widely from single very highspeed sessions to numerous low speed sessions,depending on use cases, applications, and devices. In
addition, for development using a B2B2X (businessto-business-to-X) business model, it is also importantto provide services rapidly and flexibly according toservice partners’ needs.
We are researching and developing networks thathave the capabilities to respond to various needs. In
the NetroSphere concept [1] devised by NTT laboratories, network functions are separated into components and modules, and services are provided rapidly
and flexibly by combining necessary components andmodules. A key element to realize this concept is anedge router system under development that accommodates many users, classifies the service to be
applied to user traffic, and distributes traffic flexibly
1

to appropriate service functions in the cloud. Thissystem also has a larger transfer capacity, higher
power efficiency, and higher reliability than current
edge routers, which will reduce network costs. Furthermore, this system has a policy control*1 functionthat enables new services to be provided, leading toincreased revenue.
2. Cost reduction and reliability enhancement ofedge router system
2.1 L
 arger transfer capacity and higher power
efficiency
The traffic volume in communication networks has
increased as the demand for video transfer services
has grown, video definition has become higher, andthe number of mobile devices has rapidly increased.
A high performance edge router is needed in order toefficiently respond to such traffic increases. That is,
the edge router system requires a large transfer capacity that supports 100-GbE interfaces and accommodates many users. This will make it possible to*1 Policy control: Control of user traffic such as filtering, priority
control, and rate limiting, according to rules (policy) defined forcertain kinds of traffic.

NTT Technical Review

Regular Articles

Inter-chassis
redundancy
control function

MaintenanceBackup(regularly)

Distinguishing
between active
routers and standby
routers formaintenanceCore
network

Service control
function

Config. of #N+1
for maintenanceDisguising active andstandby routers as thesame router using thesame logical IF toquickly recoverservices

Config. of #2
for service control

Config. of #2
for maintenance

Edge router#1 (Active)

Setting configuration to arouter targeted for activemode

Config. of #N+1
for maintenanceConfig. of #2
for service control

Edge router#2 (Active)

Edge router#N (Active)

Switchover
Access network

Config. of #2
for service control

Edge router#N+1 (Standby)

Edge router#N+m (Standby)

Sharing standby
routers among
multiple active routers

IF: interface

Fig. 1. Inter-chassis redundancy with shared standby routers.

greatly reduce system costs and power consumption
per bit or per user. Furthermore, when fewer edge
routers are required in a network, we can expect a
decrease in the amount of maintenance work that
depends on the amount of equipment, for example,
file updates and replacement of broken modules.
2.2 Inter-chassis redundancy
Meanwhile, the effect of a system down failuresuch as a double failure of hardware modules
becomes significant since systems can accommodate
more subscribers than ever before. The edge routersin the system being developed have redundant module structures and redundancy in line interfaces.
Moreover, this system has an inter-chassis redundancy function to achieve higher reliability. In theinter-chassis redundancy scheme as illustrated inFig. 1, standby routers are prepared as a substitute foran active router; they provide services in the eventthat the active router goes down. To achieve high reliVol. 14 No. 5 May 2016

ability and low cost, this scheme can adopt the N+mredundant architecture in which N active routersshare m standby routers, which reduces the standby
router costs.
An inter-chassis redundancy control function
implemented outside of the edge routers orders theedge routers to perform a switchover. The configurations of all active routers which include the information about system settings and per user settings are
retained in the inter-chassis redundancy control function. In the event of a switchover, the control function
sets the configuration of the active router in thestandby router that is targeted for the active mode.
Since the information for maintenance or systemmanagement is related to each apparatus, it is preferable to use the information that was originallyassigned to the standby router. In contrast, the information for providing services such as the interfaces tothe service control servers is the information that was
assigned to the original active router. This enables
2

Regular Articles

Service provider’s network/Internet

Servicefunction

Servicefunction

Logicaltunnels

Servicefunction

Separating service functions for:− Quick addition of new functions− Flexible use of resources
Servicefunction

Distribution ofservice traffic

Servicesubscription
information

Distribution ofservice traffic

Edge routerServicesubscription
information
Edge routerFig. 2. Distribution of traffic to service functions.

services to be restored quickly from a system down
fault.
3. Flexible addition of service functions
3.1 S
 eparation of service function from routerfunctionIn the existing service edge router, a specific packet
transfer function of each service such as the IP (Internet protocol) telephone service is implemented in arouter chassis. This makes it possible to reduce thenumber of chassis as well as the capital expenditure.
However, this implementation results in a complex
packet process function in the edge router. Therefore,
the number of development tasks required for deploying new service functions increases since sufficientverification is necessary to avoid degradation of otherservices. Furthermore, operation and maintenancework such as file updates for some services might
affect the other services.
We aim to introduce an isolated architecture inwhich service functions are separately deployed froman edge router chassis in order to enable new functions to be added easily and quickly. This isolatedarchitecture may be especially effective in cases
where rapid introduction of new functions is impor3

tant and where a centralized deployment is efficient,
for example, when service demand is low or highly
varied. Using network functions virtualization(NFV)*2 technology to implement service functions
makes it possible to reduce costs for equipment byusing low-priced general-purpose servers and sharing
resources among services.
We plan to gradually implement an isolated deployment and virtualization of service functions. Several
factors should be considered in order to determine theappropriate deployment of service functions, forexample, improvement of packet processing performance in NFV and the required quality level of services. For instance, more service functions can be
virtualized as the packet processing speed in virtualized functions increases and delay-sensitive services
are deployed in local datacenters.
3.2 D istribution of service traffic to appropriate
service functions
To deploy service functions separately, the edge
router classifies traffic according to its service andtransfers the classified traffic to the appropriate service functions, as shown in Fig. 2. The edge router*2 NFV: The deployment of network functions as software runningin a virtual machine environment on a general-purpose server.

NTT Technical Review

Regular Articles

Service provider’s network/Internet
Safe sites

Servicefunction

Policy control server(PCRF)

Traffic
distribution(by app.)

DPI
(TDF)

PolicyHarmful sites− Sharing DPI function among multipleedge routers− Distributing traffic to DPI function byedge routers

URL filtering
PolicyRate

Time
Packetcount
Edge router(PCEF)

Bandwidthrate control
PolicyTraffic
distribution(by 5-tuple)
PolicyTraffic
distribution(by 5-tuple)

Edge router(PCEF)

PolicyPCEF: policy and charging enforcement function
PCRF: policy and charging rules function
TDF: traffic detection function

Fig. 3. Policy control and traffic distribution functions.

determines a service function in which an incoming
packet is routed according to the service subscriptionof the user and the layer 3 and layer 4 (L3/L4) information in the packet. Then the edge router transfers
the packet to the service function appropriate to eachpacket using logical tunnels that enable transit
regardless of the destination address in the packet. In
the future, we aim to achieve more flexible packet
transit according to services using service chaining
technology, which easily controls packets and enables
them to be transferred through all of the necessary
service functions.
4. Function to provide value-added services
4.1 P olicy control by the edge router and deep
packet inspection
To provide value-added services flexibly, this system has a policy control function that can control
traffic according to individual user needs and a trafficVol. 14 No. 5 May 2016

distribution function that uses high-layer informationsuch as the kind of application, as shown in Fig. 3.
Examples of use cases include bandwidth rate control
depending on the amount of transferred packets, URL(Uniform Resource Locator) filtering to prevent
access to harmful sites, and packet transfer to an
apparatus that has a value-added function such as an
optimizer to convert the bandwidth of video traffic.
To provide such service processes, this system has anapplication based control function that uses deep
packet inspection (DPI)*3 in addition to the L3/L4
based control in the edge router. The edge router and
DPI have the capabilities to measure the volume oftraffic, filter packets, limit bandwidth, and distribute
traffic to service functions according to a policy rulepredefined for each user. In the future, we aim to support dynamic setting of policy rules to change the*3 DPI: Packet analysis and examination function using higher-layer
information in a packet payload.

4

Regular Articles

Service provider’s network/Internet

Various sensors in network
Flow collector
Network security
controller

Detailed
traffic analysis
DPI

Flow
information
Edge router

Cleaning attack
traffic from outside

Business user

Blocking attack
traffic to network
Detecting/blocking/cleaning
attack traffic

Attacker/infected
terminal

Fig. 4. Network security using edge router and DPI.

service control depending on the state of use.
4.2 Efficient use of DPI function
The cost per data transfer amount tends to be highin the DPI function since this function performs acomplex process, that is, analyzing the payloads ofpackets. Placing DPI inline requires a very high performance DPI process in order to handle all transferred packets, which of course increases costs. This
problem can be solved by sharing the DPI function
among multiple edge router functions when only partof the traffic is the object of a DPI process such as an
optional value-added service or an analysis of specified traffic. The edge router function distinguishes thepackets to be processed in DPI and distributes themto the DPI function through logical tunnels. With thisscheme, the cost of the DPI function can be reduced
since the required throughput of the DPI function is
only for processing selected traffic.

5

5. Extension to provide additional value
NTT Network Service Systems Laboratories isstudying an extension to provide more value-added
network services by combining the above-mentionedfunctions, that is, flexible distribution of service traffic and application based classification and control
using the DPI function, with other external network
functions.
5.1 Network security
One research subject is network security (Fig. 4).
The edge router filters packets to prevent unnecessary
access to network equipment, giving it a network
security capability. However, it is important toimprove network security further as we get closer tothe year 2020 since security threats are increasing
year by year. We are studying features of the edge
router system to be utilized in conjunction with various security devices, while still accommodating
users. Specifically, those features involve sending
traffic measurement data to an analyzer in order to
NTT Technical Review

Regular Articles

Service provider’s network/Internet

Cache
serverTranslation
server
Wi-Fi accesspointsEasy customization of services
according to user needs

DPI

Brick Block Box
Event site

Providing closed
communication among
specific user groups
via user authenticationEdge
routerAbility to use the same
service regardless of theaccess methods

StationFig. 5. Flexible service gateway technology (Brick Block Box).

detect security attacks and threats, distributing trafficsuspected of being attack traffic to security devices
by order of a security controller, and also blocking
attack traffic by order of a security controller. We aim
to contribute to improving network security measuresin order to respond to new security threats efficiently
and flexibly.
5.2 S ervice gateway for temporary demand for aservice
We are researching a flexible service gateway
named the Brick Block Box that can cope with temporary demand for services during events or with thevarious needs of corporations. We use vCPE (virtualcustomer premises equipment) technology, in which
a communication device originally located in a user
home is virtualized, to implement the Brick BlockBox, which is deployed on a general-purpose serverin the cloud. The Brick Block Box has location-freeconnectivity using secure tunnels and does notdepend on a particular kind of access technology. It
also features closed communication within a usergroup and access control so that only members of agroup can access service functions in order to provide
services that can easily be customized, as shown in

Vol. 14 No. 5 May 2016

Fig. 5.
6. Future prospects
To achieve a network that can respond flexibly andefficiently to the needs of collaboration partners,
NTT Network Service Systems Laboratories is developing the edge router introduced in this article. Thisedge router can flexibly distribute traffic to separately
deployed service functions and perform flexible policy control per user or per application. Our aim is toprovide more added value; therefore, we plan toenhance functions in order to achieve a more secure
network and provide flexible services in combination
with the service gateway. We are also researchingways to modularize and virtualize the edge functionin the NetroSphere concept and working on developing a more flexible, highly reliable, and efficient
future network.
Reference[1]“Feature Articles: The NetroSphere Concept—Breathing New Life
into Carrier Networks,” NTT Technical Review, Vol. 13, No. 10, 2015.
https://www.ntt-review.jp/archive/2015/201510.html

6Regular Articles

Masaaki Omotani

Director, Senior Research Engineer, Supervisor, Transport Service Platform Innovation Project, NTT Network Service Systems Laboratories.
He received his B.E. and M.E. in electricalengineering from the University of Tokyo in
1990 and 1992. He joined NTT Switching Systems Laboratories in 1992, where he conducted
research on traffic control in asynchronous transfer mode (ATM) networks and developed ATM
switching systems. He worked in the plant planning department of NTT EAST from 2002 to2005 and the planning department of NTT Network Service Systems Laboratories from 2006 to
2008. He has since been developing service edge
router systems. He is a member of the Institute ofElectronics, Information and Communication
Engineers (IEICE).

Takeshi Osaka

Senior Research Engineer, Architect, Transport
Service Platform Innovation Project, NTT Network Service Systems Laboratories.
He received his B.E. and M.E. in electricalengineering from The University of ElectroCommunications, Tokyo, in 2001 and 2003. He
joined NTT Network Service Systems Laboratories in 2003 and studied carrier-grade VoIP network architecture, edge router systems, andpacket processing architecture. During 2005–
2011, he developed the C-BGF (core bordergateway function) and I-BGF (interconnect
BGF) architecture for NTT’s Next Generation
Network (NTT-NGN). He is currently investigating and evaluating the Data Plane Development
Kit, Open Dataplane, and Network Processor for
BNG (Broadband Network Gateway) and DPI.
He is a member of IEICE.

Ichiro Kudo

Senior Research Engineer, Architect, Transport
Service Platform Innovation Project, NTT Network Service Systems Laboratories.
He received a B.E. in electrical engineering in1998 and an M.E. in informatics in 2000 from
Kyoto University. He joined Business Communications Headquarters of NTT WEST in 2000 andworked on the construction of an IP network connecting financial institutions. He joined NTT
Information Sharing Platform Laboratories in
2004, where he studied a carrier-grade NAT/
firewall for VoIP networks. During 2008–2011,
he promoted NTT-NGN and an IPv6 Internet
access service using NTT-NGN. He is currently
investigating and developing network security
technology using the edge router, DPI, and security controller for the next-generation NTTNGN.

Akiko Kuboniwa

Researcher, Transport Service Platform Innovation Project, NTT Network Service Systems
Laboratories.
She received her B.E. and M.E. in engineeringfrom University of Tsukuba, Ibaraki, in 2006 and2008. Since joining NTT in 2009, she has been
engaged in research and development of edge
router systems, DPI architecture, and IP-VPN
gateway systems. She is a member of IEICE.

7

Chiharu Morioka

Research Engineer, Transport Service Platform
Innovation Project, NTT Network Service Systems Laboratories.
She received a B.E. in electrical and electronics
engineering from Sophia University, Tokyo, in
1990. Since joining NTT, she has been working
on subjective quality assessment and traffic control methods. She is a member of the Institute ofElectrical and Electronics Engineers (IEEE).

Taizo Yamamoto

Senior Research Engineer, Architect, Transport
Service Platform Innovation Project, NTT Network Service Systems Laboratories.
He received a B.E. in civil engineering from
Osaka University in 1996 and an M.E. in infrastructure engineering from the University ofTokyo in 1998. He joined Business Communications Headquarters, NTT Kansai Branch, in 1998
and worked on the construction of a patient guidance system for a university hospital. He also
developed video communication systems at NTT
Resonant Inc. He is a member of IEICE.

Yuta Watanabe

Senior Research Engineer, Transport Service
Platform Innovation Project, NTT Network Service Systems Laboratories.
He received his B.E. and M.E. from Waseda
University, Tokyo, in 2002 and 2004. He joined
NTT Network Service Systems Laboratories in2004 and engaged in the study of multi-layer
network optimization technology for carriergrade networks. During 2008–2012, he joined
the software development team of ForCES at
NTT Advanced Technologies Corporation, andalso participated in the Internet Engineering Task
Force (IETF) interoperability test event (IETF
RFC 6984) as the implementation leader ofNTT’s software. Since 2013, he has been conducting an architectural study of inter-chassis
redundancy and investigating server coordinationand other functions of service edge routers for
introduction to commercial networks. He is amember of IEICE and IEEE.

Akira Misawa

Director, Transport Service Platform Innovation Project, NTT Network Service Systems
Laboratories.
He received his B.E., M.E., and Ph.D. in electronics engineering from Hokkaido University in
1988, 1990, and 2016. He joined NTT in 1990,
where he has been engaged in research on photonic switching systems, optical cross-connect
systems, and router system architecture. He iscurrently a director of research on edge nodearchitecture. He is a member of IEEE Communications Society and IEICE, from which hereceived the 1997 Young Engineers Award.

NTT Technical Review

Regular Articles

Tsukasa Okamoto

Vice President, Project Manager, Transport
Service Platform Innovation Project, NTT Network Service Systems Laboratories.
He received his B.E. and M.E. in mechanicalengineering from the University of Tokyo in
1987 and 1989. He joined NTT Telecommunication Networks Laboratory in 1989 and studied
ATM network performance and quality of service (QoS) design. During 1992–1995, he wasactive in telecommunication standardization in
the Telecommunication Standardization Sectorof the International Telecommunication Union,
especially in creating ATM performance recommendations. During 1996–2001, he was in theplant planning department at an NTT operatingcompany, where he was involved with the first
installation of IP networks in NTT. During
2002–2011, he studied next generation networkarchitectures and led the evolution of commercial
IP networks at NTT. Since 2012, he has beendeveloping IP transport systems including service edge routers and DPI systems. He is amember of IEICE.

Vol. 14 No. 5 May 20168

Global Standardization Activities

Standardization Activities at W3C
TPAC 2015
Kiyoshi Tanaka, Shigeru Fujimura, Shinji Fukatsu,
Tetsuro Tokunaga, Fumihisa Hamamura, Saki Homma,
Ryosuke Aoki, and Takahiro Matsumoto
Abstract

The World Wide Web Consortium Technical Plenary / Advisory Committee Meetings Week 2015
(W3C TPAC 2015) was held from October 26 to 30, 2015 in Sapporo, Japan, marking the first TPAC
meeting held in this country. This article reports on W3C TPAC 2015 with a focus on NTT Group standardization activities in TPAC.
Keywords: W3C TPAC, digital signage, blockchain, vertical writing mode

1. Introduction

2. Standardization activities at TPAC

The World Wide Web Consortium (W3C) is aninternational industry-academia joint consortium
whose members aim to standardize and promote web
technologies. It holds an annual meeting in the formof the Technical Plenary / Advisory Committee Meetings Week (TPAC), which includes both a plenary
session and standardization meetings by Working
Groups (WGs) over a one-week period. TPAC 2015
brought together 580 participants from 14 countries
at the Sapporo Convention Center in the largest-ever
TPAC meeting. Various events were also held in addition to the usual technical meetings. One such event
was a special panel discussion featuring Sir Tim
Berners-Lee, inventor of the World Wide Web andfounder of W3C, Dr. Vinton Cerf, a Transmission
Control Protocol (TCP)/Internet Protocol (IP)designer known as one of the fathers of the Internet,
and Professor Jun Murai of Keio University, who isreferred to as the Internet Samurai. Demonstrations
related to digital signage, television (TV), the Web ofThings*1, and automotive topics were also presented
by more than 25 member companies in a speciallyestablished demonstration area.

The W3C WGs normally hold discussions via mailing lists and pursue their work individually. TPAC
enables these WGs to come together for face-to-facemeetings [1]. Furthermore, as in TPAC meetings inrecent years, breakout sessions were held in an
unconference style on the day of the plenary session
so that participants themselves could propose discussion themes and participate in any other sessions theyfound interesting. The discussion themes proposed
were quite diverse, including those issues currently
faced by W3C and technical areas that W3C should
address in the future. At TPAC 2015, up to 13 sessions were held simultaneously in each of four time
periods (each lasting 60 or 90 minutes), resulting in a
total of 50 sessions.
The following sections introduce the areas in which
NTT has been making large contributions to standardization, namely, digital signage, blockchain
application technology, and vertical writing mode.

1*1 Web of Things: A term expressing the approach or architecture todeveloping services and applications using the Internet of Things
through the use of web technology.

NTT Technical Review

Global Standardization Activities

Table 1. WGs related to API proposals for digital signage.
API

Related WG

Autopilot API
Power Management API(reboot and sleep for browser or terminal)

None

Web NTP Client API(time synchronization)

Collaboration with Multi-device Timing
Community Group (under study)

System Context(system information acquisition and control)

None

System Events(system event acquisition)

SysApps WG in relation to service workers

Rich presentation API
Multiple Resources for Video(multi-source support)

HTML WG(further study of requirements needed)

Multicast Video Playback(multicast support)
Access Control of External Storage(external storage support)None(further study of requirements needed)

Double Buffering API(content preload)

SysApps WG in relation to service workers

3. Digital signage
The Web-based Signage Business Group (BG) wasestablished in April 2012 based on a proposal fromJapan. Its role is to promote the standardization ofdigital signage in W3C. This BG was formed to study
the feasibility of general digital signage that would
use web technology to control various types of displays connected to the Internet, or in other words,
web-based signage. Discussions to date have resulted
in the creation of a document providing use cases andrequirements and profile documents (core, basic
media, storage, etc.) for configuring a signage player
on a web browser using HTML5*2.
More recently, the establishment of a new WG for
discussing problems and promoting standardization
toward a web-based digital signage service was proposed. This WG would be primarily concerned withapplication programming interfaces (APIs) essential
to the operation of such a service. At TPAC 2015,
NTT organized a breakout session that was attended
by a wide range of participants and clarified discussion conditions with existing WGs on the API proposals listed in Table 1 in relation to signage requirements, toward establishing this new WG. After thesession, discussions on API selection were held at the
BG meeting with the aim of recommending a new
WG, and a general consensus on establishing a new
WG was obtained in the BG.
Agreement was also reached at the BG meeting on
Vol. 14 No. 5 May 2016

promoting formal cooperation with the International
Telecommunication Union-Telecommunication
Standardization Sector (ITU-T) based on a liaison
document issued from ITU-T Q14/16*3 seeking collaboration and document provision from W3C. We
expect coordinated standardization between W3C
and ITU-T on web-based signage to progress as wemove forward.
4. Blockchain application technology
NTT also held a breakout session on the concept ofcontent distribution utilizing blockchain technology[2], which is currently being researched at NTT Service Evolution Laboratories. The purpose of this session was to foster discussions with experts involvedin standardization on the affinity of blockchain technology with the web and to raise the presence of NTT
in the field of blockchain application technology.
The blockchain technology discussed here is a coretechnology of Bitcoin*4. It enables the formation of a
*2 HTML5: The fifth major revision of hypertext markup language,
the standard programming language for creating web pages andapplications.
*3 ITU-T Q14/16: A Question established for studying digital sign­
age systems and services at ITU-T.
*4 Bitcoin: A peer-to-peer (P2P) virtual currency based on open
protocol. As a particular feature, it secures trust without the existence of a centralized administering authority by having Bitcoin
participants perform a competitive verification procedure whenever new currency is issued.

2

Global Standardization Activities

publicly open and highly trusted ledger (blockchain)that records all transactions. In parallel with thegrowing interest in Bitcoin, the robust and generalpurpose characteristics of blockchain technology
have led to discussions on its application to fields
other than virtual currencies. NTT Service Evolution
Laboratories aims to invigorate the content distribution in the web by applying blockchain technology
and clarifying the right of content ownership and theissuing of user licenses. The main point of thisresearch and development effort is to establish amethod that firmly ties content itself to rights information on the blockchain since the latter manages
only rights information.
In the discussions at this breakout session, we
received support from many of the participants forour concept to apply blockchain technology to enablea highly trusted exchange of rights even on the OpenInternet, which was close to the Open Web*5 concept
advocated by W3C. However, applications of blockchain technology are so novel that they have not yet
found widespread acceptance. At the conclusion ofthis session, a proposal was made to continue discussions on a voluntary basis within a Community
Group, for example.
5. Vertical writing mode
The day before TPAC 2015 got underway, a Japanese Industry Meetup event was held between members of the Cascading Style Sheet (CSS)*6 WG andrepresentatives of Japanese publishing, e-books, andother related industries as a standardization activity
concerning Japanese text layout on the web. The participants at this event discussed text composition in
the Japanese writing system such as vertical writing
mode, horizontal-in-vertical setting, ruby, sidelines,
and line breaking rules. The content of these discussions was also reflected in the TPAC 2015 meetings.
As a result, W3C members from around the world
gained a better understanding of the importance andnecessity of the vertical writing mode and otheraspects of Japanese text layout on the web.
6. Support of TPAC 2015 meetings
As a Gold Sponsor, the NTT Group supported
TPAC 2015, and in order to contribute to the smooth
operation of the meetings, it installed digital signage
for guidance within the venue and provided a network
infrastructure for participants.

3

6.1 Digital information signage
NTT Service Evolution Laboratories in cooperation with related companies offered and voluntarily
constructed a digital information signage system toprovide guidance and information to participants atthe venue. Past TPAC meetings had no digital information signage, so this was the first attempt at providing such a service.
This digital information signage consisted of two
types of displays—a landscape type oriented in thehorizontal direction like a standard monitor, and a
portrait type in which the monitor’s screen is rotated
90 degrees. The displays were installed in five locations within the venue. The role of the portrait signage was to provide meeting-related informationsuch as schedules, rooms, and sponsors, while that of
the landscape signage was to provide information
related to the surrounding area such as weather
reports, subway service details, and Sapporo sightseeing information and promotional videos.
For both the landscape and portrait displays, this
digital information signage was constructed as webbased signage that uses a web browser equipped witha TV receiver as a signage player while keeping in
mind future standardization trends. Some portions of
the content were presented as vertical text, as shownin Fig. 1, which is an example of a vertical writingmode other than that used in books and documents.
Additionally, in the presentation of video content,
we succeeded in delivering full high-definition (HD)video by employing a multicast technique using avirtual network to minimize the load on the actual
network. The delivery of full HD video did not seemto hamper participant use of the network and is therefore expected to achieve high viewer satisfaction.
Despite the fact that digital information signage had
not previously been provided at a TPAC venue, it waswell received within the venue and was put to good
use as an information provision service for participants. We therefore feel that it was fully accepted andable to contribute to the success of TPAC 2015.
In addition to the above support, the NTT Strategic
Business Development Division prepared a participant-oriented application (app) for meeting support
that was also used by participants at TPAC 2015
(Fig. 2). This app included functions enabling users
to check the meeting schedule, find venue access and*5 Open Web: The idea that specifications of standard technologies
for configuring the web should be open to the public and thatanyone should be able to use them royalty-free.
*6 CSS: A language for expressing the style of a web page.

NTT Technical Review

Global Standardization Activities

Fig. 1. Digital information signage for guidance.

by checking sightseeing information and otherdetails. In the future, we would like to make this app
even more convenient by linking it with other services and adding more post-meeting support functions with an eye to promoting excursions in the surrounding area.

Fig. 2. App for participants.

venue floor maps, retrieve information using a beacon*7, and get sightseeing information for post-meeting activities. An examination of the usage data
revealed that this app was used in the morning tocheck access to the venue and that day’s schedule andin the evening to plan activities for the following day
Vol. 14 No. 5 May 2016

6.2 Venue network infrastructure
As a TPAC 2015 network sponsor, NTT Communications provided network connections including theInternet connection at the venue, and as a Network
Operation Center (NOC) member, it constructed andoperated a Wi-Fi environment within the venue.
At this TPAC, digital information signage and exhibition booths of member companies were installed
for the first time, which involved various network
requirements. Here, to provide a problem-free infrastructure for meetings and exhibits, a 1-Gbit/s dedicated line was used for the main line, and three
FLET’S lines were prepared as backup and demonstration lines. The final number of activated accesspoints exceeded 90. As a result, NTT Communications was able to provide a stable network during
TPAC 2015. A questionnaire given at the end of thisTPAC indicated that it received a high evaluation of*7 Beacon: A mechanism for receiving radio signals emitted byBluetoothTM using a receiver on the mobile-terminal side to obtain location information.

4

Global Standardization Activities

Fig. 3. R-env cloud-based visual programming environment.

8.26 points out of 9 for its network infrastructure.
7. Developer meetup in Sapporo
A Developer Meetup in Sapporo was also held as aparallel event to TPAC 2015 in the evening on the firstday of meetings at the same facility. This event wasplanned and carried out by NTT Communications incooperation with W3C to provide a discussion forum
for W3C members and related individuals coming for
TPAC from around the world and for web developers
in the Sapporo area. It included talks on the latest web
developments by six Japanese and overseas web engineers with real-time English/Japanese interpretation
provided by a volunteer staff member. In addition,
exhibition booths set up in cooperation with companies and municipalities in the Sapporo area werebustling with activity as many participants came tovisit.
NTT Service Evolution Laboratories provided an
exhibition on R-envTM [3], a cloud-based integrated
development environment for creating interactive
robot services. The aim of this exhibit was to promote
the use of R-env and to obtain feedback from web
engineers on the technology. Specifically, R-env has
a cloud-based visual programming environment on a
web browser for developing and executing applications that combine multiple devices including robots,
sensors, and gadgets (Fig. 3). Participants in the
meetup event were given demonstrations on creating
an application linking multiple devices as well as an
explanation and demonstration of a method for add5

ing a new device to R-env. In this latter method, adevice can be registered with R-env by simply using
the protocol of the JSON*8 data-interchange format
over WebSocket*9. Many web engineers wanted to try
using R-env and expressed the hope that it would
become open source software. The evaluations of theparticipants were taken into account, and the plan is
to create more opportunities to enable even more web
engineers to try out R-env. NTT Communications,
meanwhile, presented two demonstrations on
WebRTC*10 and received many useful commentsfrom participants.
References
[1]

[2]

[3]

S. Fujimura, T. Yamada, K. Tanaka, T. Kawabata, M. Ihara, K. Kom­atsu, and A. Akutsu, “W3C TPAC 2014 Report and HTML5 Recommendation,” NTT Technical Review, Vol. 13, No. 5, 2015.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201505gls.html
A. Akutsu, K. Hidaka, M. Inoue, N. Ito, T. Yamaguchi, S. Fujimura,and A. Nakadaira, “Developing Technologies for Services that
Deliver the Excitement of Games Worldwide,” NTT Technical
Review, Vol. 13, No. 7, 2015.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201507fa2.html
T. Matsumoto, N. Matsumura, T. Hosobuchi, T. Mochizuki, H. Yoshikawa, and T. Yamada, “Cloud Based Integrated Development Environment for Creating Interaction Robot Services,” IEICE Technical
Report, CNR2015-13, Vol. 115, No. 283, pp. 33–36, 2015 (in Japanese).

*8 JSON: A type of data description format.
*9 WebSocket: An API that performs full-duplex communication
with a server from JavaScript to exchange messages.
*10 WebRTC: An API that achieves P2P communication of video,
voice, and data between browsers.

NTT Technical Review

Global Standardization Activities

Kiyoshi Tanaka

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received a B.E., M.E., and Ph.D. in communication engineering from Osaka University
in 1992, 1994, and 2005. He joined NTT in 1994
and since then has been engaged in researching
video-on-demand systems and metadata-related
interactive video systems and services, especiallythose related to IPTV and digital signage services. His current interests include standardizationof IPTV and digital signage. He is a member ofthe W3C Web-based Signage BG, Web and TV
Interest Group (IG), the Institute of Electronics,
Information and Communication Engineers(IEICE), and the Human Interface Society inJapan.

Shigeru Fujimura

Research Engineer, Natural Communication
Project, NTT Service Evolution Laboratories.
He received an M.S. in information science andtechnology from the University of Tokyo in 2005
and joined NTT the same year. Since then, he has
been engaged in research on web mining and web
engineering, especially on effective methods for
implementing web applications. He is a memberof the W3C Web and TV IG.

Fumihisa Hamamura

Associate Manager, NTT Strategic Business
Development Division.
He received a B.A. in environmental information from Keio University, Kanagawa, in 2005
and joined the NTT Group the same year. He has
recently been engaged in business development
and is currently working on new business development in the MICE (meetings, incentives, conferences, and exhibitions) market and the sports
tech and sports tourism markets.

Saki Homma

Research Engineer, Department of Technology
Development, NTT Communications Corporation.
She received an M.S. in information scienceand technology from the University of Tokyo in2011 and joined NTT Communications the same
year. In 2011–2012, she was engaged in developing a big data analysis engine. Since 2013, she
has been researching leading-edge web technology related to real-time communication.

Ryosuke Aoki

Shinji Fukatsu

Senior Research Engineer, Natural Communication Project, NTT Service Evolution Laboratories.
He received a B.E., M.E., and Ph.D. in engineering from Osaka University in 1997, 1999,
and 2002. He joined NTT Cyber Solutions Laboratories in 2002. His research interests include
human interfaces, virtual reality, web technologies, metadata, and IPTV services. He is a member of IEICE.

Researcher, Networked Robot and Gadget
Project, NTT Service Evolution Laboratories.
He received a B.E. in engineering and an M.S.
and Ph.D. in information sciences from Tohoku
University, Miyagi, in 2005, 2007, and 2014.
Since joining NTT in 2007, he has been conducting research in the areas of human computerinteraction, multimodal interaction, motor learning, and service design.

Takahiro Matsumoto

Tetsuro Tokunaga

Research Engineer, Natural Communication
Project, NTT Service Evolution Laboratories.
He received a Bachelor of Liberal Arts andMaster of Informatics from Yokohama National
University, Kanagawa, in 2002 and 2004. He
joined NTT in 2004 and has been studying web
technology. His current research interests include
mastering web technology to achieve humancentered services. He was a co-developer on ateam that participated in 2003 in the youth division in the Exploratory IT Human Resources
Project (The MITOH Program) initiated by the
Information-technology Promotion Agency ofJapan. He is a member of the Information Processing Society of Japan and the Database Society of Japan.

Vol. 14 No. 5 May 2016

Researcher, Networked Robot and Gadget
Project, NTT Service Evolution Laboratories.
He received an M.S. in information and computer science from Keio University, Kanagawa,in 2012 and joined NTT the same year. Since
then, he has been engaged in research on humanrobot interaction. He is a member of IEICE
Information and Systems Society TechnicalCommittee on Cloud Network Robotics.

6External Awards
Achievement Award
Winner: Shuto Yamamoto, NTT Network Innovation Laboratories
Date: February 19, 2016
Organization: The Institute of Electronics, Information and Communication Engineers (IEICE)
For the research and development of ultra-high-capacity multilayer-integrated transport system.
Interaction 2016 Best Paper Award
Winner: Naomi Yamashita, NTT Communication Science Laboratories; Hideaki Kuzuoka, University of Tsukuba; Keiji Hirata, Future
University Hakodate; Takashi Kudo, Osaka University; Eiji Aramaki,
Kyoto University; and Kazuki Hattori, University of Tsukuba
Date: March 2, 2016
Organization: The Information Processing Society of Japan (IPSJ)Interaction 2016 executive committee
For “Mimamo-Mate: A Web Application for Family Caregivers to
Track Their Caregiving Activities.”
Published as: N. Yamashita, H. Kuzuoka, K. Hirata, T. Kudo, E.
Aramaki, and K. Hattori, “Mimamo-Mate: A Web Application for
Family Caregivers to Track Their Caregiving Activities,” Proc. of
IPSJ Interaction 2016, Tokyo, Japan, Mar. 2016.
2015 Temperature Award for the Best Puzzle
Winner: Hsin-Ni Ho, NTT Communication Science Laboratories
Date: March 11, 2016
Organization: Editorial board of Temperature

Laboratories
Date: March 17, 2016
Organization: IEICE
For “Outdoor to Indoor Path Loss Characteristics for 8 to 37 GHz
Band.”
Published as: M. Inomata, W. Yamada, M. Sasaki, and T. Onizawa,“Outdoor to Indoor Path Loss Characteristics for 8 to 37 GHz Band,”Proc. of the 2015 IEICE Society Conference, BS-1-8, Sendai, Miyagi, Japan, Sept. 2015 (in Japanese).
Young Researcher’s Award
Winner: Naotaka Shibata, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE
For “Performance Evaluation of Mobile Fronthaul Transmission
Employing Ethernet-based TDM-PON.”
Published as: N. Shibata, T. Tashiro, S. Kuwano, J. Terada, and A.
Otaka, “Performance Evaluation of Mobile Fronthaul Transmission
Employing Ethernet-based TDM-PON,” Proc. of the 2015 IEICE
General Conference, B-8-15, Kusatsu, Shiga, Japan, Mar. 2015 (inJapanese).
Young Researcher’s Award
Winner: Hiroshi Takahashi, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE

For “Red... How does it feel?”
Young Researcher’s Award
Winner: Farhan Mahmood, NTT Network Technology Laboratories
Date: March 17, 2016
Organization: IEICE
For “Design of DC-AMN with Considering Longitudinal Conversion Loss.”
Published as: F. Mahmood, K. Okamoto, H. Tatemichi, K. Takaya,
and N. Kuwabara, “Design of DC-AMN with Considering Longitudinal Conversion Loss,” Proc. of the 2015 IEICE Society Conference,
B-4-34, Sendai, Miyagi, Japan, Sept. 2015 (in Japanese).
Young Researcher’s Award
Winner: Minoru Inomata, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE
For “Prediction Method for Propagation Loss Characteristics in
NLOS Street Microcell Environment.”
Published as: M. Inomata, N. Ohmaki, W. Yamada, M. Sasaki, T.
Sugiyama, M. Mizoguchi, K. Kitao, and T. Imai, “Prediction Method
for Propagation Loss Characteristics in NLOS Street Microcell Environment,” Proc. of the 2015 IEICE General Conference, B-1-28,
Kusatsu, Shiga, Japan, Mar. 2015 (in Japanese).
Young Researcher’s Award
Winner: Minoru Inomata, NTT Access Network Service Systems

1

For “Splice Loss Measurement Using End-reflection Assisted
Brillouin Analysis.”
Published as: H. Takahashi, K. Toge, F. Ito, and C. Kito, “Splice
Loss Measurement Using End-reflection Assisted Brillouin Analysis,” IEICE Tech. Rep., Vol. 113, No. 265, OFT2013-22-37, pp.
39–42, Oct. 2013 (in Japanese).
Young Researcher’s Award
Winner: Hiroshi Takahashi, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE
For “Brillouin-based PON Monitoring Using Frequency-swept
Pump Pulse.”
Published as: H. Takahashi, C. Kito, K. Toge, and T. Manabe,“Brillouin-based PON Monitoring Using Frequency-swept Pump
Pulse,” Proc. of the 2015 IEICE Society Conference, B-13-17, Sendai, Miyagi, Japan, Sept. 2015 (in Japanese).
Young Researcher’s Award
Winner: Takayoshi Mori, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE
For “Compensation of Inter-channel Signal Quality Difference in
Mode-division-multiplexed Transmission.”
Published as: T. Mori, T. Sakamoto, M. Wada, T. Yamamoto, F.

NTT Technical Review

External Awards

Yamamoto, and K. Nakajima, “Compensation of Inter-channel Signal Quality Difference in Mode-division-multiplexed Transmission,”
IEICE Tech. Rep., Vol. 115, No. 201, OCS2015-33, pp. 13–18, Aug.
2015 (in Japanese).
Young Researcher’s Award
Winner: Takayoshi Mori, NTT Access Network Service Systems
Laboratories
Date: March 17, 2016
Organization: IEICE
For “Experimental Evaluation of Inter-modal Fiber Nonlinearity in
MDM Transmission.”
Published as: T. Mori, T. Sakamoto, M. Wada, A. Urushibara, T.
Yamamoto, and F. Yamamoto, “Experimental Evaluation of Intermodal Fiber Nonlinearity in MDM Transmission,” Proc. of the 2015
IEICE Society Conference, B-13-28, Sendai, Miyagi, Japan, Sept.
2015 (in Japanese).
Young Researcher’s Award
Winner: Yui Yoshida, NTT Network Innovation Laboratories
Date: March 17, 2016
Organization: IEICE
For “An Efficiency Evaluation of Dispersed Storage Systems
Based on Irregular Flat XOR Codes.”Published as: Y. Yoshida, T. Nakachi, D. Shirai, and T. Fujii, “An
Efficiency Evaluation of Dispersed Storage Systems Based on Irregular Flat XOR Codes,” Proc. of the 2015 IEICE Society Conference,
A-20-6, Sendai, Miyagi, Japan, Sept. 2015 (in Japanese).
Young Researcher’s Award
Winner: Hideya So, NTT Network Innovation Laboratories
Date: March 17, 2016
Organization: IEICE
For “Suppression Design of Undesired Reflections from Frequency-selective Surfaces for Multiband Antenna Employing FSSs” and“A Proposal on Undesired Radiation Suppression Technique Using
Adaptive Control of Transmission Plate for Distributed Array Antenna Systems in Mobile Environment.”
Published as: H. So, A. Ando, T. Sugiyama, and K. Cho, “Suppression Design of Undesired Reflections from Frequency-selective
Surfaces for Multiband Antenna Employing FSSs,” Proc. of the 2015
IEICE General Conference, B-1-149, Kusatsu, Shiga, Japan, Mar.
2015 (in Japanese); H. So, K. Suzaki, D. Goto, Y. Suzuki, and F.
Yamashita, “A Proposal on Undesired Radiation Suppression Technique Using Adaptive Control of Transmission Plate for Distributed
Array Antenna Systems in Mobile Environment,” Proc. of the 2015

Vol. 14 No. 5 May 2016

IEICE Society Conference, B-3-18, Sendai, Miyagi, Japan, Sept.
2015 (in Japanese).
Young Researcher’s Award
Winner: Yasuhiro Mochida, NTT Network Innovation Laboratories
Date: March 17, 2016
Organization: IEICE
For “A Proposal of UI Construction Using WebSocket for REMOCOP.”
Published as: Y. Mochida, D. Shirai, and T. Fujii, “A Proposal of UIConstruction Using WebSocket for REMOCOP,” Proc. of the 2015
IEICE Society Conference, B-11-13, Sendai, Miyagi, Japan, Sept.
2015 (in Japanese).
Young Researcher’s Award
Winner: Mitsuteru Yoshida, NTT Network Innovation Laboratories
Date: March 17, 2016
Organization: IEICE
For “Blind Chromatic Dispersion Estimation Using Spectral Symmetry of Real-valued Signals.”
Published as: M. Yoshida, K. Yonenaga, and A. Hirano, “Blind Chromatic Dispersion Estimation Using Spectral Symmetry of Real-valued Signals,” Proc. of the 2015 IEICE Society Conference, B-10-23,
Sendai, Miyagi, Japan, Sept. 2015 (in Japanese).
Maejima Hisoka Award
Winner: Kazuhide Nakajima, NTT Access Network Service Systems Laboratories; Shigeru Tomita, NTT Advanced Technology
Corporation; and Izumi Sankawa, NTT Electronics Corporation
Date: March 18, 2016
Organization: Tsushinbunka Association
For the research and development of hole-assisted optical fibers.
SUEMATSU-Yasuharu Award
Winner: Yoshihide Tonomura, NTT Service Evolution Laboratories
Date: June 2, 2016 (presentation ceremony)
Organization: IEICE
For the research and development of high-efficiency forward error
correction (FEC) codes and the contribution to ISO/IEC MPEG/
MMT standardization.
Published as: Y. Tonomura, D. Shirai, T. Nakachi, T. Fujii, and H.
Kiya, “Layered Low-density Generator Matrix Codes for Super HighDefinition Scalable Video Coding System,” IEICE Trans. Fundamentals, Vol. E92-A, No. 3, pp. 798–807, 2009.

2

Papers Published in Technical Journals andConference Proceedings
Performance Evaluation of SSB Transmission of DFTsOFDM Using Multi-level BPSK through Nonlinear HPA
M. Umehira, S. Nihei, H. Fusayasu, T. Miyajima, S. Takeda, J.
Mashino, and T. Sugiyama
Proc. of VTC2015-Spring (the 2015 IEEE 81st Vehicular Technology Conference), Glasgow, UK, May 2015.
This paper describes performance evaluation results of single sideband (SSB) transmission of discrete Fourier transform spreading
orthogonal frequency division multiplexing (DFTs-OFDM) using
multi-level binary phase shift keying (BPSK) through a nonlinear
HPA (high power amplifier). SSB-DFTs-OFDM is generated byeliminating the USB (upper side band) or LSB (lower side band) in
the frequency domain after DFT spreading of multi-level BPSK signals. Thus, a Hilbert transformer is not necessary to generate SSB
signals. The simulation results show that SSB-DFTs-OFDM achieves
almost the same peak-to-average power ratio (PAPR) and adjacentchannel power leakage (ACLP) as conventional DFTs-OFDM andlower PAPR and ACLP than OFDM.

Study on Multiple Stream Transmission by Using Multiple
Polarizations
M. Arai, T. Seki, and N. Shinohara
Proc. of the 45th European Microwave Conference, pp. 1359–
1362, Paris, France, September 2015.
In this paper, we propose a new method for achieving multiple
stream transmission by using multiple polarizations. Using polarization switching sequences corresponding to streams enables interferences to be canceled and multiple streams to be made because thepolarizations are symmetrical regardless of antenna spacing. Therefore, the method is effective when antenna spacing is small. Numerical analysis revealed that the channel capacity per unit area achieved
with our method increases in proportion to the number of streams.

Cavity-less On-chip Optomechanics Using Excitonic Transitions in Semiconductor Heterostructures
H. Okamoto, T. Watanabe, R. Ohta, K. Onomitsu, H. Gotoh, T.
Sogawa, and H. Yamaguchi
Nature Communication, Vol. 6, Article no. 8478, October 2015.
The hybridization of semiconductor optoelectronic devices andnanomechanical resonators provides a new class of optomechanicalsystems in which mechanical motion can be coupled to light without
any optical cavities. Such cavity-less optomechanical systems interconnect photons, phonons, and electrons (holes) in a highly integrable platform, opening up the development of functional integrated
nanomechanical devices. Here we report on a semiconductor modulation-doped heterostructure-cantilever hybrid system, which realizes efficient cavity-less optomechanical transduction through excitons. The opto-piezoelectric back-action from the bound electronhole pairs enables us to probe excitonic transition simply with a
sub-nanowatt power of light, realizing high-sensitivity optomechanical spectroscopy. Detuning the photon energy from the exciton resonance results in self-feedback cooling and amplification of the thermomechanical motion. This cavity-less on-chip coupling enables
highly tunable and addressable control of nanomechanical resona-3

tors, allowing high-speed programmable manipulation of nanomechanical devices and sensor arrays.

Selective-layer-free Blood Ionogram Using a 0D Nanotransistor Biosensor
R. Sivakumarasamy, K. Nishiguchi, A. Fujiwara, and N. Clément
Proc. of the 28th International Microprocesses and Nanotechnology Conference (MNC 2015), 11D-2-4, Toyama, Japan, November
2015.
We show that a 0D nanotransistor in liquid has unique properties
that enable ion selectivity without the need for ion-selective layers.
We performed a blood ionogram for various ions with a 1.5 mm × 1.5
mm silicon chip.

Electron Emission Using Multilayered-graphene/SiO2/Si
Heterodevice Driven by Low-voltage Supply in Low Vacuum
D. Yoshizumi, K. Nishiguchi, Y. Sekine, K. Furukawa, A. Fujiwara, and M. Nagase
Proc. of MNC 2015, 13P-11-104L, Toyama, Japan, November
2015.
In this study, taking advantage of this low electron scattering rate,
we demonstrate a new application: a multilayered-graphene/SiO2/Si
heterodevice for electron emission. Since electrons gain energy during their tunneling event through the heterostructure, electrons are
emitted from the device with a low-voltage supply in low vacuum.

Scalable and Locality-aware Distributed Topic-based Pub/
Sub Messaging for IoT
Y. Teranishi, R. Banno, and T. Akiyama
Proc. of IEEE GLOBECOM 2015, San Diego, CA, USA, December 2015.
Topic-based pub/sub (TBPS) messaging plays an important role inbuilding event-driven Internet of Things (IoT) applications. In IoTapplications, scalability and locality-awareness are important properties that help to achieve low-latency message delivery and efficient
usage of network resources. However, none of the existing distributed
TBPS methods can simultaneously achieve a sufficient level of bothproperties. This paper proposes a new TBPS overlay method called‘Skip Graph-based TBPS with Locality-Awareness’ (STLA), which
extends existing Skip Graph-based TBPS messaging by adding
locality-awareness. STLA determines the order of the keys on a Skip
Graph overlay network according to the network hierarchy structureusing ‘locality-aware topic keys’ (LATK). Using ‘split-forward
broadcasting’ (SFB) with LATK, the locality-awareness can be dramatically improved.

Perceptual Illusions for Multisensory Displays
T. Amemiya
Proc. of IDW 2015 (the 22nd International Display Workshops),

NTT Technical Review

Papers Published in Technical Journals and Conference Proceedings

pp. 1276–1279, Ohtsu, Shiga, Japan, December 2015.
Human perceptual properties have been applied for designing
multisensory display technologies. This paper overviews the sensoryillusion-based approach we have used to create a force display thatelicits illusory continuous force sensation by presenting asymmetricvibrations and a self-motion display based on a cross-modal effectbetween visual and tactile motion.

Orthogonalized Directional MIMO Transmission Using
Higher Order Mode Microstrip Antennas
M. Arai, T. Seki, K. Hiraga, K. Sakamoto, H. Toshinaga, and T.
Nakagawa
IEICE Transactions on Communications, Vol. E99-B, No. 1, pp.
48–57, January 2016.
We propose a spatial division method using orthogonal directivities formed by using higher order modes of rectangular microstrip
antennas. Each of them is formed by one antenna element so thatchannels are orthogonalized only by antennas. We verify antennaradiation characteristics by using higher order mode microstrip
antennas and confirm that orthogonal directivities are obtained withthem. Measurement of two-stream transmission reveals that themethod achieves almost the same channel capacity as that of an
eigenmode-beamforming method because of the high multiplexing
gain it achieves.

A DNA Aptamer Recognising a Malaria Protein Biomarker
Can Function as Part of a DNA Origami Assembly
M. Godonoga, T.-Y. Lin, A. Oshima, K. Sumitomo, M. S. L. Tang,
Y.-W. Cheung, A. B. Kinghorn, R. M. Dirkzwager, C. Zhou, A.
Kuzuya, J. A. Tanner, and J. G. Heddle
Scientific Reports, Vol. 6, No. 21266, February 2016.
DNA aptamers have potential for disease diagnosis and as therapeutics, particularly when interfaced with programmable moleculartechnology. Here we have combined DNA aptamers specific for themalaria biomarker Plasmodium falciparum lactate dehydrogenase(PfLDH) with a DNA origami scaffold. Twelve aptamers that recognise PfLDH were integrated into a rectangular DNA origami, andatomic force microscopy demonstrated that the incorporated aptamers preserve their ability to specifically bind the target protein. Captured PfLDH retained enzymatic activity, and protein-aptamer binding was observed dynamically using high-speed AFM. This work
demonstrates the ability of DNA aptamers to recognise a malaria
biomarker whilst being integrated within a supramolecular DNA
scaffold, opening new possibilities for malaria diagnostic approaches
based on DNA nanotechnology.

Sequence Alignment as a Set Partitioning Problem
M. Nishino, J. Suzuki, S. Umetani, T. Hirao, and M. Nagata
Journal of Natural Language Processing, Vol. 23, No. 2, pp. 173–
194, March 2016 (in Japanese).
Sequence alignment, which involves aligning elements of two
given sequences, occurs in many natural language processing (NLP)
tasks such as sentence alignment. Previous approaches for solving
sequence alignment problems in NLP can be categorized into twogroups. The first group assumes monotonicity of alignments; thesecond group does not assume monotonicity or consider the continuity of alignments. However, for example, in aligning sentences ofparallel legal documents, it is desirable to use a sentence alignment

Vol. 14 No. 5 May 2016

method that does not assume monotonicity but can consider continuity. Herein, we present a method to align sequences where block-wisechanges in the order of sequence elements exist. Our method formalizes a sequence alignment problem as a set partitioning problem,
which is a type of combinatorial optimization problem, and solves the
problem to obtain an alignment. We also propose an efficient algorithm to solve the optimization problem by applying column generation.

Dense Space Division Multiplexed Transmission over Multicore and Multimode Fiber for Long-haul Transport Systems
T. Mizuno, H. Takara, K. Shibahara, A. Sano, and Y. Miyamoto
Journal of Lightwave Technology, Vol. 34, No. 6, pp. 1484–1493,
March 2016.
In this paper, we review recent progress on space division multiplexed (SDM) transmission and our proposal of dense SDM (DSDM)
with more than 30 spatial channels toward capacities beyond
petabit/s. Furthermore, we discuss the requirements for realizing
long-haul DSDM transport systems using multicore and/or multimode fiber, including power and space efficient amplification
schemes, the use of fibers with large effective areas and transmission
lines with low intercore crosstalk, low differential mode delay(DMD), and low mode dependent loss (MDL). Graded index heterogeneous 12-core × 3-mode fiber with low crosstalk, low DMD and
low MDL, parallel multiple-input and multiple-output signal processing, low mode-dependent gain Erbium-doped fiber amplifiers,
and MDL equalization technologies are significant in terms ofextending the reach of multicore and multimode transmission. We
review our long-distance transmission experiment on polarizationdivision multiplexed 16-quadrature amplitude modulation signalingover 12-core × 3-mode fiber.

New Indoor Accelerated Weathering Condition forApproximating Molecular Weight Distribution of Outdoorexposed LDPE
T. Miwa, Y. Takeshita, Y. Akage, M. Watanabe, M. Takaya, and T.
SawadaJournal of Chemistry and Chemical Engineering, Vol. 9, No. 8, pp.
481–493, March 2016.
LDPE (low-density polyethylene) photodegraded through various
accelerated weathering tests has molecular weight distribution curves
unlike that attained through outdoor exposure. The authors therefore
developed new weathering test conditions based on the existing
accelerated weathering test using a xenon arc lamp. Samples of
LDPE were photodegraded using various accelerated weathering test
conditions and outdoor exposure. The physical properties and chemical structures of the photodegraded samples were studied through atensile test, infrared spectroscopy, and gel chromatography. The
authors found that the molecular weight distribution curve of a sample photodegraded using a xenon lamp at a higher BPT (black panel
temperature) (73 °C) was more similar to that of an outdoor-exposed
sample than that of a sample photodegraded at the standard BPT (63°C). It is considered that a higher temperature accelerates radical
recombination, consequently recreating molecular enlargement similar to the outdoor-exposed sample. Multiple regression analysis using
newly introduced Mp (peak-top molecular weight) and Mw (weightaverage molecular weight) as explanatory variables was conducted,
which successfully enabled the authors to provide a simple explanation for the decrease in polymer tensile strength.

4Papers Published in Technical Journals and Conference Proceedings

32-core Dense SDM Unidirectional Transmission of PDM16QAM Signals Over 1600 km Using Crosstalk-managed
Single-mode Heterogeneous Multicore Transmission LineT. Mizuno, K. Shibahara, H. Ono, Y. Abe, Y. Miyamoto, F. Ye, T.
Morioka, Y. Sasaki, Y. Amma, K. Takenaga, S. Matsuo, K. Aikawa,
K. Saitoh, Y. Jung, D. J. Richardson, K. Pulverer, M. Bohn, and M.
YamadaProc. of OFC (Optical Fiber Communication Conference andExhibition) 2016, Th5C.3, Anaheim, CA, USA, March 2016.
We demonstrate 32-core dense space-division multiplexed (SDM)
unidirectional transmission of PDM-16QAM (polarization-divisionmultiplexed 16-quadrature amplitude modulation) 20-WDM (wavelength-division multiplexed) signals over 1644.8 km employing a
low-crosstalk single-mode heterogeneous 32-core fiber in a partial
recirculating-loop system.

5

A Sparsity Managed Adaptive MIMO Equalization for Fewmode Fiber Transmission with Various Differential Mode
DelaysD. Lee, K. Shibahara, T. Kobayashi, T. Mizuno, H. Takara, A.
Sano, H. Kawakami, T. Nakagawa, and Y. Miyamoto
Journal of Lightwave Technology, Vol. 34, No. 8, pp. 1754–1761,
April 2016.
It is often observed that various differential mode delays (DMDs)
coexist in single multi-core fiber and/or few-mode fiber transmission.
From a multi-input and multi-output (MIMO) equalization perspective, this indicates that the optimum equalization tap length for each
multi-core and/or multi-mode signal varies according to its DMD.
Correspondingly, complex calculation to find each optimum tap
length is necessary to obtain satisfactory performance. This paper
presents a new adaptive MIMO equalization method to deal withvarious DMDs while avoiding such complex calculation. The method
uses the same tap length for all multi-core and/or multi-mode signals
according to the maximum DMD to reduce the calculation cost.

NTT Technical Review