NTT Technical Review

November 2018 Vol. 16 No. 11
View from the Top

Kazuhiro Yoshizawa, President and Chief Executive Officer, NTT DOCOMO

Front-line Researchers

Ryuichiro Higashinaka, Senior Distinguished Researcher, NTT Media Intelligence Laboratories

Feature Articles: Further Exploring Communication ScienceShift to New Dimensions—Further Initiatives to Deepen Communication ScienceSpeakerBeam: A New Deep Learning Technology for Extracting Speech of a Target Speaker
Based on the Speaker’s Voice Characteristics



Network Reliability Optimization by Using Binary Decision Diagrams

Ukuzo—A Projection Mapping Technique to Give Illusory Depth Impressions to
Two-dimensional Real Objects

Cross-media Scene Analysis: Estimating Objects’ Visuals Only from Audio

Measuring, Understanding, and Cultivating Wellbeing in the Age of Technology

Regular Articles
 High-speed Avalanche Photodiodes toward 100-Gbit/s per Lambda Era
Global Standardization Activities

Telecom Infra Project—Its Structure and Activities

Information

Event Report: NTT Communication Science Laboratories Open House 2018

Short Reports

Ultrahigh-speed Integrated Circuit Capable of Wireless Transmission of 100 Gbit/s in the
300-GHz Band

External Awards/Papers Published in Technical Journals and Conference Proceedings

View from the Top

Toward Co-creation of New Value
through a Receptive Attitude toOutside Ideas—NTT DOCOMO’s“Declaration Beyond” Medium-term
Strategy Embodied in Six
Declarations
Kazuhiro Yoshizawa
President and Chief Executive Officer,
NTT DOCOMO
Overview

With its sights firmly on the 5G (fifth-generation mobile communications
networks) era, NTT DOCOMO has achieved a gigabit-capable mobile network by increasing its effective download speed by 1.5 times over the previous year. Breaking away from its traditional image of a mobile and smartphone company, NTT DOCOMO has strived to improve its long-term corporate value and to create a safe and secure, comfortable, and richer society,for which it has gained a high reputation, including earning a No. 1 ranking
in total score in the 2018 Toyo Keizai CSR (Corporate Social Responsibility) Ranking. We asked Kazuhiro Yoshizawa, President and Chief Executive
Officer of NTT DOCOMO, about his outlook and aspirations based on theMedium-Term Strategy 2020 known as “Declaration beyond.”
Keywords: 5G, artificial intelligence, co-creation

Medium-Term Strategy 2020 “Declaration
beyond” aiming for sustainable growth—Mr. Yoshizawa, NTT DOCOMO is said to be undergoing a business transformation. How are things
going?
Business operations are on the right track.
Competition in the mobile market is becoming
increasingly severe, so we are working day in and day
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

out to gain the favor of our customers. For example,
customer satisfaction is high with the “docomo with”
billing plan, and our subscriber base is increasing
steadily. However, we cannot become complacentwith these results. Our Medium-Term Strategy 2020
“Declaration beyond” announced in April 2017 is
aimed at establishing a solid business platform while
we earnestly address the ever-changing needs of society and return more value to customers (Fig. 1). We
will continue to challenge new frontiers with an eye
1View from the Top

Declaration 1

Declaration 4

Market leader

Industry creation

offering visible benefits
and convenience

jointly pursuinginnovations with 5G

Declaration 2
Style innovation

Declaration

for a richer future
for all

Declaration 5
Solutionco-creation
to solve social issues

Declaration 3

Declaration 6

Peace of mind andcomfort support

Partner businessexpansion

always by ourcustomers’ side

leveraging
business platforms

Fig. 1. Medium-Term Strategy 2020 “Declaration beyond.”

to 2020 and beyond. We aim to create a richer futurewith fifth-generation mobile communications systems (5G) by providing surprising and exciting services exceeding our customers’ expectations and by
co-creating new value with our business partners. At
present, we are steadily moving into the execution
phase.
As its services expand beyond communications,
NTT DOCOMO is redefining the meaning of “customer.” Up to now, we have focused all of our energy
on increasing the number of our mobile subscribers,
but with the penetration rate of mobile phones now
exceeding 100% in Japan, we are making a shift froma customer foundation based on “subscribers” to one
based on “membership.” Here, a “member” is someone who uses NTT DOCOMO services regardless ofwhether that person is an NTT DOCOMO mobile
subscriber. In other words, we seek to maximize thevalue that we can offer, even to customers who may
be mobile subscribers of other companies but who
use NTT DOCOMO services such as “d POINT”(loyalty points program), “d CARD” (credit card),
and “dmarket” (content delivery portal).
Specifically, as stated in “Declaration 1: Market
leader” of “Declaration beyond,” we aim to lead themarket in providing benefits, convenience, and surprise, by integrating and enhancing services, billing
plans, and loyalty points programs. Similar ecosysNTT Technical Review

Vol. 16 No. 11 Nov. 2018tems already exist in other companies’ services, but
we aim to expand the number of “d POINT” partners
to more than 300 companies by fiscal year 2020 andto develop “d POINT” as one of Japan’s largest loyalty points programs.
We are also busy with a variety of initiatives in relation to artificial intelligence (AI). For example, in
May 2018 we launched the “my daiz” AI agent service—one of the nine challenges proclaimed in“Declaration 2: Style innovation.” On assuming theoffice of president, I announced my goal to achieve
the ultimate personal agent that blends in perfectlywith people’s lives, and I can say that “my daiz” istruly the precursor of this agent. This service anticipates behavior using information obtained from acustomer’s smartphone operations and assists thecustomer in choosing actions that make for a smoother day. For example, “my daiz” can anticipate andprovide information that the customer will likely
need such as information on a delay in the running ofa train that the customer always uses. Looking forward, we want to be able to anticipate customer
behavior to make personalized and optimized proposals. To this end, we continue to refine AI, which takeson the role of a “mind,” to provide surprise andexcitement beyond the customer’s expectations.
In addition, we plan to use 5G with augmented reality, virtual reality, AI, and the Internet of Things (IoT)
2

View from the Top

to achieve “experience innovation,” “life style innovation,” and “work style innovation.” For example, bypartnering with content providers and broadcasters,
we plan to place multiple cameras at a venue anddeliver a live program such as a sports event and concert by 4K/8K video using NTT’s “Kirari!” immersive telepresence technology and other video processing technologies. In this way, we hope to provide
customers in remote locations with the sense of reality as if they were present at the venue. Highpresence, high-resolution, wide angle, and multiple
simultaneous transfers constitute an area that makes
full use of 5G. We plan to exploit these features toprovide customers with truly exciting experiences.
—It appears that many initiatives are being launchedthat will transform the world.
That’s true, and 5G in particular will be a driving
force. With the aim of offering the 5G experiencesooner, NTT DOCOMO will deliver a “pre-service”
at the stadiums that will be used in the Rugby World
Cup to be held in September 2019 in Japan. Buildingon these achievements, we plan to commence commercial services of 5G in the spring of 2020.
As stated in “Declaration 4: Industry creation,”
NTT DOCOMO will contribute to the further development of society and industry by leveraging the 5Gfeatures of high speed and large capacity, low latency,
and massive device connectivity to expand the possibilities for business in collaborations with partnercompanies. I believe that 5G will be a major pillar ofour digital transformation, which will lead to (1)improved business operations, that is, improved productivity, (2) radically enhanced user-interface/userexperience designs, and (3) the creation of totally

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

new, revolutionary services. We aim to achieve thesetransformations through 5G.
I remember well, when 3G and LTE (Long-Term
Evolution) services were being launched, it was often
said, “Are these high-speed communications reallynecessary?” But as it turned out, the sudden spread ofsocial media, streaming video, and smartphones have
today made those high-speed communications an
absolute necessity. The approach taken here was tobuild the network first and to catch up with services
later. In the case of 5G, however, our goal is to provide 5G-era services from day one, that is, to launchboth the 5G network and 5G services at the sametime. For this reason, co-creation with a wide rangeof partners is essential.
To facilitate this co-creation of new 5G-era services
with diverse partners, we provide the DOCOMO 5G
Open Partner Program at no charge. More than 1800
companies and organizations have already come toparticipate in this program, which features workshops that include lectures and lively discussions
among partners on creating new 5G services. We also
provide the DOCOMO 5G Open Lab technologytesting environment in which 5G experimental basestation equipment can be used without charge. One
site was set up in Tokyo in April and another in Osakain September, and one is scheduled to open in
Okinawa in December.
Furthermore, with the aim of deploying specific 5G
services, we are conducting a wide variety of trials—
more than 70—with partners at 5G Trial Sites thathave been set up in the Tokyo Waterfront City district
and Tokyo Skytree® neighborhood. We expect that
promoting co-creation with diverse partners at an
early stage in this way will support the rapid creationof new services in the 5G era.
Let me give you two examples of these trials. The
first trial involves the remote operation of construction machinery, and we are conducting this jointlywith Komatsu Ltd. In this trial, we mount 5G terminals on construction machinery such as hydraulicexcavators and bulldozers and have a remote operator
control the machinery while watching video of it via
5G. Through this trial, we are studying the feasibilityof performing accurate and efficient on-site work andwork management from a remotely located office bychecking on-site conditions in real time through NTT
DOCOMO 5G connections.
The second trial entails remote medical services
that we conducted in collaboration with the Wakayama
Prefecture and Wakayama Medical University. In thistrial, we first transmitted images of ultrasound-image
3

View from the Top

diagnostic apparatus (echo) and MRI (magnetic resonance imaging) systems, or 4K images of the affectedarea of the patient’s body to the university’s Regional
Medical Support Center. We then asked a medical
specialist at the university to make a diagnosis via 4K
teleconference that could support medical treatment
at the rural clinic. In this way, we are trying to resolve
disparities in the provision of healthcare services, oneof the major social issues in the country.
Value creation arises from co-creation
with partners—Changes are also taking place at construction sites,
and the workers involved could be changing.
That’s right. The on-site working environment incivil engineering and construction has long been
described as “3K,” referring to the three Japanesewords kitsui, kitanai, and kiken (hard, dirty, andrisky). I’ve heard that the driver’s seat in some models
of bulldozers is not equipped with air conditioning.
For on-site work during hot and humid summers andat accident scenes or hazardous zones that people
should not enter, it will become possible to use a camera in place of the human eye and transmit high-resolution video by 5G to a remote monitor. This capability will enable remote operation of machinery in realtime as if the operator were at the actual site.
We can also consider that the skillful use of big data
can be applied to solve social problems. For example,
we are implementing the LANDLOG open IoT platform to improve productivity in the constructionindustry. LANDLOG uses IoT technology to collectdata at construction sites on dump trucks that carryearth and on worker movements and other details. It
will then analyze that data using AI so that on-site
operations can be optimized and work efficiency
improved. Our goal here is to achieve process innovation at construction sites.
Additionally, as stated in “Declaration 5: Solution
co-creation,” we are working to solve social problems
and revitalize regional economies in primary industries, education, mobility, and other fields. This is
expected to stimulate growth in Japan and create aricher society.
I can give you some examples of applications inprimary industries such as agriculture and the fishingindustry. In agriculture, we are working to reduce themanual labor associated with the day and night monitoring of rice paddies by applying a system thatmeasures the temperature and level of water in a
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

paddy and visualizes such data on the farmer’s smartphone screen. In the fishing industry, we are working
on the ICT (information and communication technology) Buoy solution system that measures ocean datasuch as water temperature and salt concentration andvisualizes those data on smartphones. This system
makes it possible to reduce the risks of relying only
on human experience and know-how and to complement such know-how with data, thereby improving
quality and efficiency in the cultivation of seaweed
and oysters.
Meanwhile, while it was only a short time ago that
services such as self-driving cars were thought to besomething only seen in movies, we now know thatsuch unbelievable services may be achievable in thereal world through technical innovation. It would not
be possible, however, to switch all cars from being
ordinary ones to self-driving ones all at once.
Whatever the matter may be, it’s simply not feasible
to make a complete change at the same point in time.
Likewise, in our initiatives for construction sites,
agriculture, and the fishing industry, it is essentialthat we apply a plan-do-check-act (PDCA) cycle any
number of times while reflecting the best idea at eachtime. Innovation never ends. I believe that repeating
the PDCA cycle is one way of fostering innovation.
—“Repetition” appears to be a straightforward
approach.
Yes it is, and management as well undergoes cyclesof improvement. Here, one must take a hard look at
the results to date, but there is no guarantee that revenues will increase by using the same method as
before. New things must always be incorporated, andyou yourself must change.
At the same time, even if the economy and social
conditions should be undergoing substantial change,
the reason for NTT DOCOMO’s existence does notchange, which I believe to be the continuous provision of new value to customers and society. At present, however, what we can do by ourselves is limited.
Accepting ideas and technologies possessed by avariety of partner companies and organizations iswhat I call receptivity, and combining the strengths ofall parties involved to create totally new value is what
I recognize to be open innovation, or in other words,
co-creation.
We call this initiative +d (plus d), which involves aneffort to achieve positive results for both a partner
and NTT DOCOMO. The +d initiative is already mak­ing progress in diverse fields. NTT DOCOMO has
4View from the Top

extensive business assets such as its mobile network,
membership base, and secure settlement system.
Combining these assets with our partners’ assets
should generate a “chemical reaction” from which we
can create new business useful to our partners andco-create new social value.
In today’s world, value creation will be limited if
we rely only on the abilities of individuals or single
companies. We must therefore collaborate with partners and bring out ideas. As part of this +d initiative,
we are trying to change the procedure for product andservice development used in the past, where salespeople meet with customers and listen to what they
have to say and pass that information on to those incharge of technology for reflection in development.
When salespeople talk to customers nowadays, theyfrequently need more technical knowledge than they
did before. Therefore, research and development(R&D) staff should also participate in the discussions
with customers from the very start so that we candirectly connect customer needs to development or
make proposals that anticipate the customer’s needs
or wishes.
The program we have developed called “Top Gun”
can reduce the time and labor involved in building abridge from corporate sales and marketing to R&D,
thereby enabling us to repeat the PDCA cycle more
rapidly. In this way, customers, corporate sales andmarketing, and R&D become a three-party collaboration that can solve problems quickly and create newvalue.
The first project of the Top Gun program was anexperimental trial of a child monitoring solution forelementary school students in Kobe City, Hyogo
Prefecture. This solution involves attaching aBluetooth Low Energy tag to students’ backpacks andchecking whether they had passed a certain locationby using sensors installed throughout the city. The
R&D team also proposed a mechanism that uses theBluetooth function of smartphones by asking people
in various occupations such as taxi drivers, insurance
salespersons, and security guards to install a special
application in their smartphones so that position
information could be collected whenever such a person and a student passed each other.
This was the system eventually deployed in Kobe
City. This made it possible to acquire a more detailed
history of movement than with any previous system.
We also applied this mechanism to systems to determine the location of baggage carts in airports, baby
carriages, and other items. In fact, we launched aservice called “Location NetTM” using this mechaNTT Technical Review

Vol. 16 No. 11 Nov. 2018

nism in October 2017. Therefore, the Top Gun program has already been implemented in more than 10
projects, including the ones mentioned here.
Having a receptive attitude unconstrained by
one’s standing or position—Mr. Yoshizawa, can you leave us with a message for
all NTT DOCOMO employees?
I would be happy to. I am very conscious of receptivity, which I think is a very important attitude tohave. I always say, “I would like you to always keep
in mind the perspective of others, regardless ofwhether the person you’re talking to is your superioror subordinate.” In other words, let’s listen carefully
to each other, on an equal footing in a humble manner. It’s easy to give instructions to a subordinate
from the position of a manager, but doing so may narrow the range of that person’s work or thoughts or
suppress initiative. Listening closely to what someone has to say can help clarify the proposer’s logic or
ideas. In such cases, I don’t hesitate to adopt the otherperson’s idea if it’s better than mine. I also don’t want
to miss out on hearing new viewpoints or ideas by
cutting off a conversation saying, “I already know allabout this.” For this reason, I try to hear proposals
directly whenever possible. Of course, I may encourage someone to make it short because of time limitations, but basically speaking, I place importance on
listening closely. I associate this attitude with our +d
initiative. One’s standing or position within the company does not matter. This is the way we should
approach our work throughout NTT DOCOMO.
Through such an attitude, I am convinced that we canshare values with our customers and gain their understanding of NTT DOCOMO’s initiatives and efforts.
5

View from the Top

Nevertheless, even with these efforts, there are few
times when business proceeds absolutely smoothly.
What happens more often than not is that one’s work
does not go to plan and proceeds in an undesirable
direction. At such times, it is important that you do
not avoid something you don’t like and that you workto overcome the difficulty. Business is like walking
along a narrow ridge with a steep cliff on both sides.
If you suddenly come to a stop or even if you fall off,
you have to face the situation calmly and think about
your next step. Additionally, even if you have beenwalking well so far, you can still tumble down thecliff if you start to walk aimlessly. Arrogance is yourenemy here. This is what I think business is all about.
Keep this in mind and approach your work as something that is enjoyable. Pessimism has no place here.
Let’s do our work filled with optimism!

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Interviewee profile
Career highlights
Kazuhiro Yoshizawa joined Nippon Telegraphand Telephone Public Corporation (now NTT) in
1979. He has been involved in the mobile phonebusiness since the dawn of the industry. He
became NTT DOCOMO Senior Vice President
and Managing Director, Corporate Sales andMarketing Department II, in 2007, Senior Vice
President and Managing Director of the Human
Resources Management Department in 2011, andExecutive Vice President, Managing Director of
the Corporate Strategy & Planning Department in2012. He became Senior Executive Vice President
in June 2014. He assumed his current position inJune 2016. Concurrently, he retains his position
as a member of the Board of Directors, which heassumed in 2011.

6

Front-line Researchers

Pursuing a Dialogue System for
Making Society More Harmonious
Ryuichiro Higashinaka
Senior Distinguished Researcher, NTT
Media Intelligence Laboratories
Overview
The spread of smartphones and artificial intelligence (AI) systems that speak means that the opportunities for general users tocome into contact with AI are increasing. Users are now expecting
improved ease of use and more new technologies from researchand development. Under these circumstances, NTT has continuously gained attention—both domestically and globally—inannouncing pioneering technologies in the fields of question
answering functions and language processing. We visited Ryuichiro Higashinaka, Senior Distinguished Researcher at NTT MediaIntelligence Laboratories, and asked him about the progress andoutlook of his research, his attitude as a researcher, and how the development of AI and a future in whichrobots and humans can talk smoothly will bring about changes in our society.
Keywords: artificial intelligence, natural language processing, question answering technology

Research on dialogue systems takes steady work
—Tell us about the research that you are currently
working on and the initiatives undertaken so far.
I joined NTT in 2001, and since then, I have been
researching language processing, artificial intelligence (AI), and dialogue systems. A dialogue system
is a technology for interacting with a computer. It is
probably easier to understand if imagined as an animated robot character like Doraemon or Astro Boy
who can talk smoothly with humans. The idea of“smoothness,” that is, “naturalness,” is the point of
my research, and for that reason it is necessary to
unravel what kind of elements a person’s conversation is made up of.
As shown by the fact that 60% of human conversation constitutes chat (casual conversation), chat is
NTT Technical Review

Vol. 16 No. 11 Nov. 2018very important. A relationship between people cannot
be established simply by having a conversation about
work. Chat serves as cushioning material that makes
it possible to know the other person’s personality,
thereby encouraging cooperative work. Let’s consider that one person in a conversation is replaced with a
dialogue system (i.e., a computer): if the computer
does not understand what kind of person the person
is, and if the person does not understand what kind ofthing the computer is like, the conversation will notgo smoothly. In my past research, though, I focusedon tasks rather than chat, so I had the idea that a dialogue between a person and a computer should be
brief. However, I came to realize the importance ofcommunication and building relationships in achieving natural conversation, so I am currently focusing
on chat. In regard to the study of an actual dialoguesystem, I’m repeating the straightforward work of7

Front-line Researchers

Detailed answer type classification
Step 2: Classify the word-answer type
into more than 100 types.
(in case of word-answer type)

Step 1: Determine word/sentence-answer type.
Word, definition, reason, association, reputation, etc.
Mountain

Document
retrievalWeb/
Twitter
Mt. Fuji, Kita-dake,Mont Blanc, Everest, K2,
Kanchenjunga, Lhotse,
Kilimanjaro

Questionanalysis

User question:“What is thehighest mountainin the world?”

Answerextraction
User

Extract answer according
to detailed answer type(named entity recognition).

Answerevaluation

Answerdisplay

System answer:“Everest”

Fig. 1. Question answering logic used by “Shabette Concier.”

observing conversations and building the system, getting people to talk with the system and checking
unnatural parts of speech, and feeding the resultsback to the system. In that sense, research on dialoguesystems is step-by-step.
—Can you introduce the research that you haveworked on specifically?
There is NTT DOCOMO’s voice-agent service
called “Shabette Concier” (talking concierge), which
was launched in March 2012 in Japan. I was in chargeof the logic used by the question answering system
supporting this service (Fig. 1). The question answering system was adopted as a centerpiece function
when Shabette Concier was upgraded in June 2012. It
recognizes and analyzes a user’s words and searches
for relevant information on the Internet and finds thebest answers within a few seconds. It was designed towork smoothly even if many users are connected at
the same time. It took about half a year for us to makethe system practical, and as the practical applicationof question answering technology—which had notbeen developed on a large scale then—it received
much attention from the research community. Incidentally, even though I was directly in charge of the
final stage of development of the function, the basic
research that formed its foundation spanned over 10
years before that stage. I don’t think that we would
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

have been successful without that basic research.
As for AI, I was involved in the project called “Cana robot get into the University of Tokyo?” led by the
National Institute of Informatics (Figs. 2 and 3). Thisis a project to get AI to solve problems and answer
questions in a university entrance exam. I was incharge of the subject of English in collaboration withjoint-research institutes. At that time, we applied language processing technology with the goal ofstrengthening the English skills of Torobo-kun (thename of the AI. Torobo stands for the University ofTokyo Robot, and kun is an honorific title in Japanesenormally used for boys). We had Torobo-kun study
English in order to take the National Center Test for
University Admissions, but I actually think that wasmore difficult than studying by oneself. In particular,
clarifying what was difficult was problematic. That
admission test is made up of a variety of questions
such as pronunciation problems, filling-the-gap problems, long-sentence reading comprehension, and listening comprehension. By utilizing dictionaries andbig data, we were able to reach a deviation score of
50.5; however, we have not yet reached the acceptance standard of the University of Tokyo. All the
researchers involved are pursuing the next step. By
solving this admission test, I hope to improve language processing technology and develop moreadvanced natural dialogue systems.
Also, in collaboration with Professor Hiroshi
8Front-line Researchers

Short-sentence
problems

Multiple-sentence
problems

Long-sentence
problems

100
90
80
70
60
50
40

Mock test

30

Benchmark

20
10

st

d

ul

or

ab

(w

/v

oc

nt

ar

G

ra

m

m

Ac

ce

Pr

on

un

ci

at

io
n
r
An
ar
es
y
s
sw
El
W /wo )
im
er
r
o
d
in
r
se
i
d
at
nt
o ng
io
n
D enc rdeiaofri
un log e se ng
u
n
l
e
e
ec
R
ct
ea
es co
io
R
O
sa mp nea din
p
le
ry
g
in
di
io
ng co
se tioco mp n su nte n
r
m
m
nc
p eh
m
R
ea reh en
ar es
s
i
e
d
z
i
R
ea ing nsi on ( atio
st
di
co on
n
a
ng
(l
m
t
co pre ife isticmms)a
pr henehsi teria
o
en
ls
n
)
(
si
on sto(e ries
Ti
di
tle
to )
as rial
s
si
gn )
m
en
t

0

Fig. 2. Scoring rate achieved by Torobu-kun in comparison with benchmark (test data created at laboratory) for various
problems in University of Tokyo entrance English test.

Ishiguro of Osaka University, I’m researching an
android that can hold a conversation. Professor Ishiguro is at the forefront of research on humanoid robots.
He found the point at which human-like robots makeus feel uncomfortable and is now pursuing that something that makes us humans. Meanwhile, I want toapproach human nature by exploring what theessence of dialogue is. In common with these efforts,
we have been conducting joint research with Professor Ishiguro such as creating “Matsukoroid,” whichlooks like the famous Japanese television personality
Matsuko Deluxe, in 2015.
Moreover, in a collaborative experiment with
Dwango Co., Ltd., I have begun work on the development of AI of “Ayase” (a character in a light-novel
series called “My Little Sister Can’t Be This Cute.”)to thereby create “Ayase AI” (Fig. 4). This project isaimed at developing AI with personality with theuser’s involvement. A certain amount of data isrequired to make a computer personality like a person; however, currently there is not enough data in thewhole world to express personality, and from theviewpoint of privacy, it is difficult to obtain personal
data. Ayase AI started from the idea that if the data are
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

not available, people should make the data. By asking
users to become Ayase and talk on the web, we cancollect fundamental data and foster the personality of
Ayase based on such data. As the amount of data
grows, the dialogue system becomes unique. Users
can feel as if they’re really talking to someone, so I
think we are getting closer to finding the essence ofdialogue.
—What is the significance of your research?
I think that the significance of my research is topursue the essence of human beings. We as humans
have elements of ourselves that we do not understand.
Humans are social creatures that cannot live by themselves, so we have developed communication skills
for the purpose of living with others. I believe that if
we can clarify that communication on a scientificbasis, humans will get closer to understanding eachother. I believe that if human mutual understanding
progresses, cooperative work will become smoother,
we will feel happier, and so on, leading to improved
quality of life. I hope to pursue my goal of developing
a dialogue system as a shared property that will make9

Front-line Researchers(a) Grammar, vocabulary, wording (completion of declarative
statement) (short-sentence problems)(b) Elimination of unnecessary sentences(multiple-sentence problems)(c) Reading comprehension (life materials)(long-sentence problems)Fig. 3. Examples of problems.

My Little Sister Can’t Be This Cute.

Ayase’sroom

Archive

Ranking

Anonymoususer
Q. Anonymous user:
What part of your body do you
wash first, Ayase?

Please take our survey

AnswerA. Ayase:
My left arm first….what kind ofquestion is that!? I’ll get you forthat!

Current number ofquestion answer pairs

Ask a question
Ask Ayase a question(50 characters maximum)

Fig. 4. Screenshot of question answering bulletin board.

NTT Technical Review

Vol. 16 No. 11 Nov. 201810

Front-line Researcherssociety more harmonious and allow us to lead better
lives.
Shifting from liberal arts to sciences, and taking
time and effort to overcome adversity—How did you end up on the path to become aresearcher?
Actually, my origins lie in the liberal arts. I thoughtthat I would like to study law at university, but I chosea university and faculty leaning towards public policy
after many twists and turns. At university, computer
education was vigorous, and we used email—which
was rare at that time—to exchange coursework andlearn programming. I fit in really well in that environment, so I ended up pursuing programming, andwhen I was in graduate school, I spent a year and ahalf as a student researcher at IBM Tokyo ResearchLaboratory (TRL). While at IBM TRL, I encounterednatural language processing, so I decided to take theresearcher’s path and entered the NTT laboratories.
Because I’m also very interested in foreign languages and have studied in the UK, when I joined
NTT, I offered to specialize in research on translation.
However, at that time, translation was not a statisticalprocess, like it is now, but a rule-based process conforming to rules such as grammar. Since the translation research I wanted to do was shifting to a commercial basis and not being tackled at basic researchlaboratories, I was assigned to a department responsible for dialogue systems. From that point on, I
decided to engage in research on dialogue for the firsttime. However, that research was really difficult, andeven seemingly simple tasks like booking a meetingroom by talking to the dialogue system were difficult.
With that difficulty in mind, I became more interested in why people can converse, and gave myself up
to research on dialogue. Around 2001, AI was still ina period of winter-like hardship, and dialogue systems were receiving little attention in a field considered to be for diehard researchers only. However,since I began working on dialogue systems back then,I have had many opportunities to present myresearch—which has been ongoing for 17 years—andthis research has started drawing attention.
—What is the driving force behind your research
activities?
The driving force behind my research is “curiosity.”
I consider that everything is interesting, so I take a
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

stance in which I never refuse what comes my way.
And since nothing can be done in one leap, I think it
is better to do a lot of experiments and find things out
one by one. Anyway, I value taking time and effort inmy pursuits. In that sense, I think of myself as acraftsman. As well as taking time, research does notnecessarily lead to successful outcomes, so I thinkthat it is enough to obtain one or two successes out of
100 attempts.
When I joined NTT, I was coming from a liberalarts background but was with many employees whocame from a science background, so I was immersedin things I did not understand. To overcome this hurdle, I worked on problems more carefully than others,
and I applied trial and error as much as possible. My
experiences of trial and error led to the present andhave built confidence in me.
—Do you have anything particularly memorable concerning your research activities?
We participated in a large-scale event held in theUSA, called “SXSW” (South by Southwest), which
combines a music festival, film festival, and interactive festival, in two consecutive years (2016 and2017) (Fig. 5). In 2016, we showcased the dialoguesystem of an android that talked in English in Professor Ishiguro’s demonstration, and it was wellreceived. In 2017, I was invited as a featured speaker
and took the podium with Professor Ishiguro. It wasa very honorable occasion, since among Japanesepeople, only very few have received such an invitation. That year, we wanted to demonstrate moreadvanced technology than what we did the previous
year, so we decided to set up a discussion between
humans and robots. We set up a situation in whichrobots outnumbered humans, based on the assumption that we might create a world in which robots area majority. We wanted to present a world in which a
lot of robots that are smarter than humans exist and to
get people to think about what human beings reallyare.
By the way, the demonstrations in 2016 and 2017
were packed with drama. In 2016, the demo system
was only completed the day before the demonstration. The system needed a fairly complicated program, which I continued to write after arrival, not tomention on the plane to the event. We finally got the
program to run when the system and the dialogue
partner were “face to face,” so to speak. Although thesituation at that time is now the stuff of legend, it was
truly a miracle that the program ran.
11

Front-line Researchers

Fig. 5. Demonstration of argumentative dialogue system at SXSW 2017 (in collaboration with Ishiguro Laboratory, Osaka
University).

On taking the podium in 2017, the system did notwork as we expected. I managed to keep talking forabout 30 minutes, and in the meantime, we madevarious adjustments to the system, and it finally
started working. And when it worked, everyone broke
into applause. Because I was in front of more than
1000 spectators, I felt frustrated when it didn’t work
right away. Even so, I was optimistic that it would
work at some point. Since I knew that dialogue systems are complicated and don’t work that easily, I
was probably just keeping my nerve.
Let’s find places of interest, things, and othermatters, and move forward continuously—How will research on dialogue systems proceed in
the future?
The level of Japanese research in this field is relatively high and has developed rapidly recently. Our
research is of course often influenced by the researchand development done by major global companies.
Nevertheless, many concepts that I thought about
have been achieved since I joined the company, and I
have a “sense of the future.” As for my goal, I’d likethe act of speaking with a dialogue system to become
an everyday event. Since AI and computers are notgood at everything, I think that it is better to divide
the tasks that humans will bear responsibility for and
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

the tasks left to computers, and that division willallow us to enter an era in which we can live a richer
life. To that end, I want to build a good relationshipbetween humans and computers (robots).
Robots can work longer than people can, and only
robots can do advanced simulation calculations. And
some tasks are better done if they are not done byhumans. From the viewpoint of privacy protection,
there are certain occasions, such as counseling, when
it is better to leave it to the robot. Recently, dialoguesystems have started to be applied in the counselingfield. Efforts to introduce dialogue systems in facilities for the elderly and to apply them with the aim ofdementia prevention are underway—which is said tobe a promising area.
However, I think that whether AI is able to respondto unknown events or whether it could acquire an ego
are major challenges. Current AIs do not possess anego, and discussions on ethics and legal matters inregard to whether an ego should be given to AI are
ongoing. Although guidelines concerning those matters are already being created, the industry has not yet
reached a consensus. On the other hand, I think that
we cannot carry out dialogue, in the true sense of theword, with things that have no ego. In the future, wewill also conduct research on giving AI an ego, andI’d like to pursue the concept of “What is a humanbeing?”

12

Front-line Researchers—Please say a few words to all young researchers.
I have been very blessed to have good co-workers
and leaders. Now that we are in a technologicalboom, the number of people working on dialoguesystems has increased accordingly, but when I started, there were few such researchers. Fortunately,
NTT had a group dedicated to research on dialogue,
and many employees were engaged in that research.
For the time being, I have been concentrating on academic activities in order to publicize the value of the
dialogue system and increase the number of likeminded researchers. We organize a symposium called
Dialogue System Symposium, and it has expanded toa large-scale event attended by 200 people. Some ofour projects have been exhibited at events in the USA
and have become internationalized. Through suchprojects, we have come to share a common sense ofvalue with researchers in various fields and withpeople in different positions, which is very meaningful.
It is currently difficult for fourth-year undergraduate students aiming to become researchers, postgraduate students in master’s programs, and new employees to start research on dialogue systems if there areno colleagues in the same field. I have tried various
things to foster the next generation in this field,
including publicizing data to reach such people, planning events, and so on. Through these kinds of activities, I’d like to continue to increase the number offellow researchers.
I think that it is better to share new concepts andideas to get closer to the truth. It is a good time to be
an AI researcher now. The status is also getting better,and compared to when there was no Internet, theamount of data is now abundant, and we have all thetools needed. We are in an era where you can pushforward with your own ideas and anyone who has theability can do anything. Since the world is becoming
borderless, aim to be top class and keep the world in
mind. Our ability is our output. There are various
ways of producing output such as getting papers published, writing programs, and implementing systems,

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

and if you leave your mark through output andresults, you will be acknowledged as a fellow
researcher. For that reason, trial and error is important. Although there are many different researchstyles, let’s find interesting things that will connect togood outputs.
Acknowledgments
Dr. Higashinaka thanks the National Center for
University Entrance Examinations, JC Educational
Institute, Inc., Takamiya Academy School Corporation, and Benesse Corporation for providing the problems used in the Torobo Project.

Interviewee profile
Ryuichiro Higashinaka
Senior Research Scientist (Senior Distinguished
Researcher), Knowledge Media Project, NTT
Media Intelligence Laboratories.
He received a B.A. in environmental information, a Master of Media and Governance, and a
Ph.D. from Keio University, Kanagawa, in 1999,
2001, and 2008. He joined NTT in 2001. His
research interests include building question
answering systems and spoken dialogue systems.
From November 2004 to March 2006, he was avisiting researcher at the University of Sheffieldin the UK. He received the Maejima Hisoka
Award from the Tsushinbunka Association in2014 and the Prize for Science and Technology of
the Commendation for Science and Technology
by the Minister of Education, Culture, Sports,
Science and Technology in 2016. He is a memberof the Institute of Electronics, Information andCommunication Engineers, the Japanese Society
for Artificial Intelligence, the Information
Processing Society of Japan, and the Association
for Natural Language Processing.

13

Feature Articles: Further Exploring Communication
Science

Shift to New Dimensions—Further
Initiatives to Deepen Communication
Science
Takeshi Yamada
Abstract

NTT Communication Science Laboratories aims to realize communication that reaches the heart,from person to person, and between people and computers. We are building fundamental theories inpursuit of the essence of people and of information, and working to create core technologies that willtransform society. This article introduces some of our initiatives to push deeper in communication science, in areas including speech and audio processing, dialogue processing, human information science,
sports brain science, and machine learning and optimization.
Keywords: artificial intelligence, communication science, corevo

1. Introduction
There have been some remarkable developments in
artificial intelligence (AI) recently. Developments indeep learning, in particular, have achieved capabilities approaching those of humans in areas such as
speech and image recognition and natural language
processing, which were once considered humanstrengths that could not be matched by computers. At
the NTT laboratories, we consider it important andnecessary to use these leading-edge technologies andapply them to the issues we are facing. However, as
these technologies spread, we must further strive toopen new and next-generation areas that are not simple extensions of earlier work, and to make boldchanges in our research themes.
We at NTT Communication Science Laboratories(CS Labs) aim to realize communication that reaches
the heart, from person to person, and between people
and computers. We are building fundamental theories
in pursuit of the essence of people and of information,
and working to create core technologies that willtransform society. Our fields of research support thefour types of AI that comprise the NTT Group’s AI
technology called corevo ®. These are Agent-AI,
Heart-Touching-AI, Ambient-AI, and Network-AI
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

[1]. As such, we have focused mainly on media processing, human information science, and data andmachine learning, and have also focused recently on
sports brain science (Fig. 1). The Feature Articles inthis issue introduce some of our initiatives to pushdeeper in these areas of communication science.
2. Speech and audio processing approaching
human capabilities
Agent-AI, a component of corevo, supports interaction between humans and computers. CS Labs isworking on speech and audio processing, image recognition, and natural language processing to provide
a platform for Agent-AI. Speech recognition underconditions where a single person is speaking into a
close-talking microphone has already matured to alevel where it is used in everyday life, due to the rapid
spread of smartphones and, more recently, devices
such as AI speakers.
Research and development (R&D) trends are now
shifting toward speech recognition with multiple
people speaking freely around a table, some distancefrom the microphone. This requires increased performance of speech recognition itself, but it is also
important to combine it with speech enhancement
14Feature Articles

Network-AI
Connect multiple AIs to collaborate andoptimize the entire social system.

Agent-AI

Ambient-AI

Support interaction between
humans and computers.

Analyze information from people,
things, and the environment and use
it for prediction and control.
Heart-Touching-AI
Interpret brain mechanisms thatmanipulate mind and body, such as
instinct and the unconscious.

Media processing

Sports brain science

Human information
science

Data and machinelearning

CS Labs’
researchfieldsCommunication science for communication that reaches the heart

Fig. 1. NTT’s four AI directions and communication science.

technologies such as denoising to remove background noise in noisy environments, and dereverberation to remove reverberation from the walls and floorof a room. A speech recognition technology from CS
Labs combining these capabilities was entered in the
3rd CHiME Speech Separation and Recognition
Challenge (CHiME-3) held in 2015, where it placed
first among 25 participating organizations [2]. We areworking to use these technologies for automaticallycreating the minutes of a meeting involving several
people, while it is still in an experimental stage.
Humans can pick out the voice of a person theywant to hear and can understand what that person issaying, in a discussion in a meeting or in a conversation during a party, when there are several people
talking or when music is playing in the background.
This is called selective listening. In the article in thisissue entitled “SpeakerBeam: A New Deep Learning
Technology for Extracting Speech of a Target Speaker Based on the Speaker’s Voice Characteristics ” [3],we introduce a technology that uses deep learning to
implement selective listening using computers.
People are also able to visualize a scene in their
NTT Technical Review

Vol. 16 No. 11 Nov. 2018mind just by listening to a sound. Cross-media scene
analysis implements this type of behavior in computers. Deep learning can be applied to various media ina unified framework, so cross-media scene analysis
based on deep learning transcends single-media processing, combines multiple types of media, and handles them in a complementary way. Cross-media
scene analysis is introduced in the article, “Crossmedia Scene Analysis: Estimating Objects’ Visuals
Only from Audio” [4].
3. Elimination of the human-AI gap
The capabilities of computers are approaching
those of humans in certain situations such as thosedescribed above, but it will take more time for AI
performance to advance beyond the complexity of thehuman brain. Nevertheless, humans are sometimes
easily fooled by telephone payment scams and thelike. This is related to certain human cognitive biases.
For example, people tend to look only for evidence
suited to their existing beliefs, hopes, and suppositions and to interpret circumstances accordingly. This
15

Feature Articles

My pets are
so clever!
This must
be my son!

Confirmation bias: people tend to lookonly for evidence suited to their existing
beliefs, hopes, and suppositions and tointerpret circumstances accordingly

ELIZA effect: people have a tendency
to arbitrarily anthropomorphize andsense intelligence from behavior thatsuperficially appears to be intelligent

Fig. 2. Examples of cognitive biases.

type of cognitive bias is called a confirmation bias[5], which explains why elderly people easily assume
a phone call asking for money is from their child anddo not notice any inconsistencies.
People also have a well-known tendency to arbitrarily anthropomorphize and sense intelligence frombehavior that superficially appears to be intelligent.
This is another cognitive bias called the ELIZA
effect, after a computer program developed at MIT(Massachusetts Institute of Technology) in the 1960s[6]. For example, people tend to think their pets are
smarter than they really are (Fig. 2). Moreover, there
are well-known examples of illusions such as thechecker shadow illusion, which effectively demonstrates the fact that humans do not see physical quantities as they are, literally, before their very eyes [7].
Effective use of such cognitive biases and illusions
in interfaces and for feedback is the key to filling the
gap between humans, who are complex and imperfect, and AI, which is currently still immature andlimited. Regarding cognitive biases, CS Labs is conducting research on dialogue processing for humanrobot interaction, extending the level from casualconversation to serious debate, and at the same time,
moving from dialogue with a single robot to that withmultiple (two) robots. For conversation with a single
person, a single robot may seem to be sufficient as acounterpart. However, multiple robots can be made toappear more intelligent to the human by dividing
roles suitably and by utilizing robot-robot interactionthat can take advantage of human cognitive biases
more effectively. Dialogue with multiple robots
seems more natural to the human and can be maintained for longer periods than that with a single robot,
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

even if there are speech recognition mistakes or the
dialogue context is lost [8].
CS Labs has also produced interfaces that use illusion, such as Buru-Navi, a device that produces anillusion of being pulled, and Hengento Projection,
which produces an illusion of dynamic motion by
projecting light onto a printed picture or photograph.
The article in this issue, “Ukuzo—A Projection Mapping Technique to Give Illusory Depth Impressions to
Two-dimensional Real Objects” [9], introduces a newtechnique that exploits human visual illusion.
4. Explaining implicit brain activities
To realize communication that reaches the heart,
CS Labs is focusing on explaining implicit brain
activity related to the basic human senses of vision,
hearing, and motion. Research on interfaces as
described above, using illusion, is also derived fromthis approach. This basic research on human information science is the Heart-Touching-AI component ofcorevo, an AI platform to interpret brain mechanismsthat manipulate mind and body, such as instinct andthe unconscious, and thereby support humans. Realizing communication that reaches the heart will
surely be possible by developing AI that touches theheart, namely, Heart-Touching-AI.
Initiatives in sports brain science have recently
become a new theme in Heart-Touching-AI [10]. This
research makes use of knowledge from brain sciencethat explains implicit brain activity, advanced information and communication technology such as wearable sensors and virtual reality, and machine learning
to train the brain to win sports games. This research
16Feature Articles

Which pathshould I take?
Exploitation
Search in more detail within therange of what is already wellunderstood.

Exploration
Search more widely bychanging the scope.

Fig. 3. Exploration-Exploitation Dilemma.

is not about making the body stronger but is focusedon finding a way to coordinate the body parts optimally, and how to control the human mental state
well. For example, on the basis of the fact that hearing
has better time resolution than vision, we have proposed a method that provides intuitive feedback bymeasuring the activity of muscles in certain parts ofthe body and converting them to sounds in real time.
Beyond sports, it is also important to support people in their daily lives in order to maximize use of theimplicit capabilities of their minds and bodies. The
article in this issue, “Measuring, Understanding, andCultivating Wellbeing in the Age of Technology” [11],introduces our challenge to identify design guidelines
for human wellbeing, which cannot be grasped qualitatively at a glance, treating it quantitatively from ahuman-scientific perspective, and improving it.
5. Machine learning and optimization
With machine learning technology, it is possible
from large amounts of data to automatically discover
patterns that even human experts would not notice.
The collective behavior of people is complex, as witha crowd of people walking in a town, but there aredefinite patterns, as each person has his or her own
typical intention. At NTT, we are conducting R&D on
technology that can predict the risks of congestion ordelays in the near future from real-time data obtainedby observing the flow of people. It can then automatically provide near-optimal on-line navigation to
avoid such risks as efficiently as possible without
impeding people in fulfilling their intentions. Here,
we use techniques such as multi-agent simulation andBayesian optimization [12].
With limited computing resources, it is often diffiNTT Technical Review

Vol. 16 No. 11 Nov. 2018cult to search for the optimal solution with a finetoothed comb (i.e., a simple exhaustive search) when
there are vast numbers of combinations. In suchcases, we are obliged to choose whether to prioritize
an Exploitation approach, in which we search in moredetail within the range of what is already well understood, or an Exploration approach, in which wechange the scope to search more widely. This is called
the Exploration-Exploitation Dilemma (Fig. 3).
Bayesian optimization is a method for efficiently narrowing down candidate solutions in a search, withconsideration for the balance between Exploitationand Exploration [13]. NTT is conducting this R&D as
a component of Ambient-AI, which provides AI as
intelligence in the Internet of Things, analyzing information from people, things, and the environment, andusing it for prediction and control.
On the other hand, there are also cases when data
structures can be devised so that all combinations canbe enumerated efficiently and a strictly optimal solution computed, even when a simple exhaustive search
is difficult. The article “Network Reliability Optimization by Using Binary Decision Diagrams” [14] inthis issue introduces examples of formerly unsolvable large-scale problems that were solved with surprising efficiency by using data structures such as
binary decision diagrams.
As Ambient-AI progresses and AI technologies areused more often in networks, multiple AIs will collaborate and optimize overall social systems, forming
Network-AI.
6. Future prospects
This article has introduced some new initiatives incommunication science at CS Labs. New initiatives
17

Feature Articles

also have accompanying risks. We cannot always
obtain the scientific results we hope for immediately.
In R&D as well, we face the Exploration-Exploitation Dilemma. We need to decide whether we shouldtake an Exploitation approach, using the latest technologies and attempting to apply them skillfully tothe problems we are facing, or give priority to Exploration, working to open up new dimensions that are
not extensions of earlier work [15]. We at CS Labs
will continue to face challenging problems, giving
priority to Exploration approaches. To be sure, weneed to be careful of cognitive biases in conducting
research, so that we do not only look for and interpret
evidence that is well suited to our own assumptions[16].
References
[1]

[2]
[3]

[4]

[5]

T. Yamada, S. Takahashi, F. Naya, T. Ikebe, and S. Furukawa, “Artificial Intelligence Research Activities and Directions in the NTT
Group,” NTT Technical Review, Vol. 14, No. 5, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201605fa1.html
NTT press release, “NTT Achieved Top Performance in a Noisy
Speech Recognition International Challenge,” Dec. 14, 2015.
http://www.ntt.co.jp/news2015/1512e/151214a.html
M. Delcroix, K. Zmolikova, K. Kinoshita, S. Araki, A. Ogawa, and T.
Nakatani, “SpeakerBeam: A New Deep Learning Technology for
Extracting Speech of a Target Speaker Based on the Speaker’s Voice
Characteristics,” NTT Technical Review, Vol. 16, No. 11, pp. 19–24,
2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa2.html
G. Irie, H. Kameoka, A. Kimura, K. Hiramatsu, and K. Kashino,“Cross-media Scene Analysis: Estimating Objects’ Visuals Only from
Audio,” NTT Technical Review, Vol. 16, No. 11, pp. 35–40, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa5.html
M. Akiyama, “Psychology of Deception and Trouble for Seniors,”

Kokumin Seikatsu, Vol. 13, pp. 1–4, 2013 (in Japanese).
J. Weizenbaum, “ELIZA—A Computer Program for the Study of
Natural Language Communication between Man and Machine,”Communications of the ACM, Vol. 9, No. 1, pp. 36–45, 1966.
[7] Website of Illusion Forum by CS Labs, Checker Shadow (in Japanese).
http://www.kecl.ntt.co.jp/IllusionForum/v/checkerShadow/ja/index.
html[8] NTT press release issued on January 1, 2018 (in Japanese).
http://www.ntt.co.jp/news2018/1801/180131b.html[9] T. Kawabe, “Ukuzo—A Projection Mapping Technique to Give Illusory Depth Impressions to Two-dimensional Real Objects,” NTT
Technical Review, Vol. 16, No. 11, pp. 30–34, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa4.html[10] M. Kashino, “Understanding and Shaping the Athlete’s Brain—NTT
Sports Brain Science Project—,” NTT Technical Review, Vol. 16, No.
3, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201803fa1.html[11] J. Watanabe, Y. Ooishi, S. Kumano, M. Perusquía-Hernández, T. G.
Sato, A. Murata, and R. Mugitani, “Measuring, Understanding, andCultivating Wellbeing in the Age of Technology,” NTT Technical
Review, Vol. 16, No. 11, pp. 41–44, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa6.html[12] F. Naya, M. Miyamoto, and N. Ueda, “Optimal Crowd Navigation via
Spatio-temporal Multidimensional Collective Data Analysis,” NTT
Technical Review, Vol. 15, No. 9, 2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201709fa5.html[13] E. Brochu, V. M. Cora, and N. de Freitas, “A Tutorial on Bayesian
Optimization of Expensive Cost Functions, with Application to Active
User Modeling and Hierarchical Reinforcement Learning,”
arXiv:1012.2599, 2010.
[14] M. Nishino, T. Inoue, N. Yasuda, S. Minato, and M. Nagata, “Network
Reliability Optimization by Using Binary Decision Diagrams,” NTT
Technical Review, Vol. 16, No. 11, pp. 25–29, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa3.html[15] J. G. March, “Exploration and Exploitation in Organizational Learning,” Organization Science, Vol. 2, No. 1, pp. 71–87, 1991.
[16] R. Nuzzo, “How Scientists Fool Themselves – and How They Can
Stop,” Nature, Vol. 526, No. 7572, pp. 182–185, 2015.
[6]

Takeshi Yamada

Vice President and Head of NTT Communication Science Laboratories.
He received a B.S. in mathematics from the
University of Tokyo in 1988 and a Ph.D. in informatics from Kyoto University in 2003. He joined
NTT Electrical Communication Laboratories in1988. He was a visiting researcher at the Schoolof Mathematical and Information Sciences, Coventry University, UK, from 1996 to 1997. He wasa group leader of the Emergent Learning and
Systems Research Group from 2006 to 2009 andan executive manager of Innovative Communication Laboratory from 2012 to 2013 at NTT Communication Science Laboratories. His researchinterests include data mining, statistical machinelearning, graph visualization, metaheuristics, andcombinatorial optimization. He is a Fellow of theInstitute of Electronics, Information and Communication Engineers and a senior member of
the Institute of Electrical and Electronics Engineers, and a member of the Association for
Computing Machinery and the Information Processing Society of Japan.

NTT Technical Review

Vol. 16 No. 11 Nov. 201818

Feature Articles: Further Exploring Communication
ScienceSpeakerBeam: A New Deep Learning
Technology for Extracting Speech ofa Target Speaker Based on the
Speaker’s Voice Characteristics
Marc Delcroix, Katerina Zmolikova, Keisuke Kinoshita,
Shoko Araki, Atsunori Ogawa, and Tomohiro Nakatani
Abstract

In a noisy environment such as a cocktail party, humans can focus on listening to a desired speaker, anability known as selective hearing. Current approaches developed to realize computational selective
hearing require knowing the position of the target speaker, which limits their practical usage. This article
introduces SpeakerBeam, a deep learning based approach for computational selective hearing based on
the characteristics of the target speaker’s voice. SpeakerBeam requires only a small amount of speech
data from the target speaker to compute his/her voice characteristics. It can then extract the speech ofthat speaker regardless of his/her position or the number of speakers talking in the background.
Keywords: deep learning, target speaker extraction, SpeakerBeam

1. Introduction
Automatic speech recognition technology has progressed greatly in recent years, thus enabling the
rapid adoption of speech interfaces in smartphones or
smart speakers. However, the performance of current
speech interfaces deteriorates severely when several
people speak at the same time, which often happensin everyday life, for example, when we take part indiscussions or when we are in a room where a television is on in the background. The main reason for thisproblem arises from the inability of current speech
recognition systems to focus solely on the voice ofthe target speaker when several people are speaking[1].
In contrast to current speech recognition systems,
human beings have a selective hearing ability (seeFig. 1), meaning that they can focus on speech spoken by a target speaker even in the presence of noiseor other people talking in the background by exploitNTT Technical Review

Vol. 16 No. 11 Nov. 2018

ing information about the characteristics of the voiceand the position of the target speaker.
Previous attempts to replicate computationally thehuman selective hearing ability used information
about the target speaker position [1]. With theseapproaches, it is hard to focus on a target speaker
when the speaker’s position is unknown or when he/she moves, which limits their practical usage.
We have proposed SpeakerBeam [2], a novel
approach to mimic the human selective hearing ability that focuses on the target speaker’s voice characteristics (see Fig. 2). SpeakerBeam uses a deep neuralnetwork to extract speech of a target speaker from a
mixture of speech signals. In addition to the speech
mixture, SpeakerBeam also inputs the characteristicsof the target speaker’s voice so that it can extractspeech that matches these characteristics. These
voice characteristics are computed from an adaptation utterance, that is, another recording (about 10
seconds long) of the target speaker’s voice.
19

Feature Articles

Ability to listen only to a target speaker by focusing on the characteristics of
his/her voice (pitch, timbre, etc.) and the direction of arrival of the soundFig. 1. Human selective hearing ability.

Target speaker
Recording of the target speaker’s
voice (about 10 seconds), used as
adaptation utterance for deriving
the characteristics of the voice ofthe target speaker

Speech mixture

Adaptationutterance

Extracted
target speakerSpeakerBeam
Example recording conditions

Fig. 2. SpeakerBeam’s selective hearing capability.

Consequently, SpeakerBeam enables the extractionof the voice of a target speaker based solely on thetarget speaker’s voice characteristics without knowing his/her position, thus opening new possibilities
for the speech recognition of multi-party conversations or speech interfaces for assistant devices.
In the remainder of this article we briefly review
conventional approaches for selective hearing. We
then detail the principles of the proposed SpeakerBeam approach and present experimental results
confirming its potential. We conclude this article withan outlook on possible applications of SpeakerBeam
and future research directions.

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

2. Conventional approaches for computational
selective hearing
Much research has been done with the aim of finding a way to mimic the selective hearing ability ofhuman beings using computational models. Most of
the previous attempts focused on audio speech separation approaches that separate a mixture of speech
signals into each of its original components [1, 3].
Such approaches use characteristics of the soundmixture such as the direction of arrival of the sounds
to distinguish and separate the different sounds.
Speech separation can separate all the sounds in a
mixture, but for this purpose it must know or be able
to estimate the number of speakers included in the
mixture, the position of all the speakers, and the20

Feature Articles

Target speaker

Adaptation Auxiliary networkutterance

Selective hearing achieved by
combining two neural networks

Speakercharacteristics
Main network
Adaptive layer
SpeechmixtureExtracted
target speaker

Fig. 3. Novel deep learning architecture developed for SpeakerBeam.

background noise statistics. These conditions oftenchange dynamically, making their estimation difficult
and thus limiting the actual usage of the separation
methods. Moreover, to achieve selective hearing, we
still need to inform the separation system which of
the separated signals corresponds to that of the target
speaker.
3. Principles of SpeakerBeam
SpeakerBeam focuses on extracting only the target
speaker instead of separating all components in the
mixture. By focusing on the simpler task of solely
extracting speech that matches the voice characteristics of the target speaker, SpeakerBeam avoids theneed to estimate the number of speakers, the position,or the noise statistics. Moreover, it can perform target
speech extraction using a short adaptation utteranceof only about 10 seconds.
SpeakerBeam is implemented by using a deep neural network that consists of a main network and an
auxiliary network as described below and shown in
Fig. 3.
(1)	The main network inputs the speech mixture
and outputs the speech that corresponds to thetarget speaker. The main network is a regularmulti-layer neural network with one of its hidden layers replaced by an adaptive layer [4, 5].
This adaptive layer can modify its parameters
depending on the target speaker to be extracted;
namely, it can change its parameters dependingon the characteristics of the voice of the target
speaker provided by the auxiliary network.
(2)	The auxiliary network is a multi-layer neural
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

network that inputs a recording of only the voiceof the target speaker (adaptation utterance) thatis different from that in the speech mixture. The
auxiliary network outputs the characteristics ofthe voice of the target speaker.
These two networks are connected to each otherand trained jointly to optimize the speech extraction
performance. Training the auxiliary network jointlywith the main network enables the system to learn
automatically from data the features that best characterize the target speaker’s voice, thus avoiding thecomplex task of manually engineering features characterizing the target speaker’s voice. Moreover, bytraining the network with a large amount of training
data covering various speakers and background noise
conditions, SpeakerBeam can learn to achieve selective hearing even for speakers that were not includedin the training data. Details of the network architecture and training procedure are explained in our published report [2].
4. Performance of SpeakerBeam
We conducted experiments to evaluate the speech
extraction performance of SpeakerBeam and its
impact on speech recognition [2]. We used a corpus
consisting of sentences read from English newspaper
articles and created artificially mixtures of two speakers. Although SpeakerBeam can work with a single
microphone, it achieves better performance whenusing more microphones. In this experiment, we usedeight microphones and combined SpeakerBeam withmicrophone array processing (i.e., beamforming).
An example of processed speech using SpeakerBeam
21

Feature Articles

Speaker A

Speaker B

SpeechmixtureExtracted
speech of
speaker A
True speechof speaker A
Processed speech with SpeakerBeam

Speech recognition accuracy (%)

Target speaker extraction from recordings of speech mixtures of two speakers

100
80
60

Speech mixture

40

Extracted speech

200Speech recognition accuracy of target speech(higher is better)

Fig. 4. Evaluation of speech extraction performance and automatic speech recognition with SpeakerBeam.

and the speech recognition accuracy obtained whenrecognizing mixtures of two speakers with SpeakerBeam (red bar) and without it (blue bar) are shown in
Fig. 4. We observed a 60% relative improvement inspeech recognition performance with SpeakerBeam.
SpeakerBeam can also be employed to improve theaudible quality. Interested readers can refer to a video[6] to appreciate the target speaker extraction performance in realistic conditions (real recordings inreverberant conditions with music in the background).

such as direction-of-arrival features.
Acknowledgment
Part of this development was achieved through a
research collaboration with Brno University of Technology in Brno, Czech Republic.
References[1]

5. Outlook
SpeakerBeam is a novel approach to perform computational selective hearing that offers several advantages compared to previous approaches. For example,
it can track a target speaker regardless of the numberof speakers or noise sources in the mixture andregardless of the speaker’s position. This opens new
possibilities for speech recognition of multi-party
conversations, speech interfaces for assistant devices
such as smart speakers, or for voice recorders andhearing aids that could focus on the speech of a target
speaker.
However, there are some issues that need to beaddressed before SpeakerBeam can be widely used.
For example, speech extraction performance degrades
when two speakers with similar voices speak at thesame time. To tackle this issue, we plan to investigate
improved target speaker characteristics that could
better distinguish speakers and to combine target
speaker characteristics with location information
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

[2]

[3]

[4]

[5]

[6]

T. Hori, S. Araki, T. Nakatani, and A. Nakamura, “Advances in Multispeaker Conversational Speech Recognition and Understanding,”
NTT Technical Review, Vol. 11, No. 12, 2013.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201312fa3.html
K. Zmolikova, M. Delcroix, K. Kinoshita, T. Higuchi, A. Ogawa, andT. Nakatani, “Learning Speaker Representation for Neural Network
Based Multichannel Speaker Extraction,” Proc. of the 2017 IEEE
Automatic Speech Recognition and Understanding Workshop (ASRU
2017), Okinawa, Japan, Dec. 2017.
S. Makino, H. Sawada, and S. Araki, “Blind Audio Source Separation
Based on Independent Component Analysis,” In: M. E. Davies, C. J.
James, S. A. Abdallah, and M. D. Plumbley (eds), Independent Component Analysis and Signal Separation—Proc. of ICA 2007, Lecture
Notes in Computer Science, Vol. 4666, Springer, Berlin, Heidelberg,
2007.
M. Delcroix, K. Kinoshita, A. Ogawa, S. Karita, T. Higuchi, and T.
Nakatani, “Personalizing Your Speech Interface with Context Adaptive Deep Neural Networks,” NTT Technical Review, Vol. 15, No. 11,2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201711fa6.html
M. Delcroix, K. Kinoshita, A. Ogawa, C. Huemmer, and T. Nakatani,“Context Adaptive Neural Network Based Acoustic Models for Rapid
Adaptation,” IEEE/ACM Trans. Audio, Speech and Lang. Proc., Vol.
26, No. 5, pp. 895–908, 2018.
SpeakerBeam video in English, https://www.youtube.com/
watch?v=7FSHgKip6vI
SpeakerBeam video in Japanese, https://youtu.be/BM0DXWgGY5A

22

Feature Articles

Marc Delcroix

Senior Research Scientist, Signal Processing
Research Group, Media Information Laboratory,
NTT Communication Science Laboratories.
He received an M.Eng. from the Free University of Brussels, Brussels, Belgium, and theEcole Centrale Paris, Paris, France, in 2003 and a
Ph.D. from the Graduate School of Information
Science and Technology, Hokkaido University,
in 2007. He was a research associate at NTT
Communication Science Laboratories from2007–2008 and 2010–2012 and then became apermanent research scientist at the same lab in
2012. His research interests include robust multimicrophone speech recognition, acoustic model
adaptation, integration of speech enhancement
front-end and recognition back-end, speechenhancement, and speech dereverberation. He
took an active part in the development of NTT
robust speech recognition systems for the
REVERB and CHiME 1 and 3 challenges, which
all achieved best performance results in the tasks.
He was one of the organizers of the REVERB
challenge 2014 and of the 2017 Institute of Electrical and Electronics Engineers (IEEE) Automatic Speech Recognition and Understanding
Workshop (ASRU 2017). He is a member of theIEEE Signal Processing (SP) Society Speech andLanguage Processing Technical Committee (SLTC). He is also a visiting lecturer at the Facultyof Science and Engineering of Waseda University, Tokyo. He received the 2005 Young Researcher Award from the Kansai section of the Acoustic
Society of Japan (ASJ), the 2006 Student Paper
Award from the IEEE Kansai section, the 2006
Sato Paper Award from ASJ, the 2015 IEEE
ASRU Best Paper Award Honorable Mention,and the 2016 ASJ Awaya Young ResearcherAward. He is a senior member of IEEE and amember of ASJ.

Katerina Zmolikova

Ph.D. student, Brno University of Technology.
She received a Bc. degree in information technology in 2014 and an Ing. degree in mathematical methods in information technology in 2016
from the Faculty of Information Technology,
Brno University of Technology (BUT), Czech
Republic. Since 2013, she has been part of the
Speech@FIT research group at BUT, where she
is currently working towards her Ph.D. degree.
Her research interests include robust speech recognition, speech separation, and deep learning.

Shoko Araki

Senior Research Scientist, Signal Processing
Research Group, Media Information Laboratory,
NTT Communication Science Laboratories.
She received a B.E. and M.E. from the University of Tokyo in 1998 and 2000, and a Ph.D. from
Hokkaido University in 2007. She joined NTT in
2000 and has since been engaged in researching
acoustic signal processing, array signal processing, blind source separation, meeting diarization,
and auditory scene analysis. She has been on theorganizing committees of several conferences,including ICA 2003 (Fourth International Symposium on Independent Component Analysis andBlind Signal Separation), IWAENC 2003 (2003International Workshop on Acoustic Echo andNoise Control), and WASPAA (IEEE Workshop
on Applications of Signal Processing to Audioand Acoustics) 2007 and 2017, and HSCMA2017(Fifth Joint Workshop on Hands-free SpeechCommunication and Microphone Arrays). She
has also served as the evaluation co-chair of the
Signal Separation Evaluation Campaign (SiSEC)
2008, 2010, and 2011. She has been a member of
the IEEE Signal Processing Society Audio andAcoustics Technical Committee (AASP-TC)
since 2014, and a board member of ASJ since2017. She received the 19th Awaya Prize from
ASJ in 2001, the Best Paper Award of IWAENC
in 2003, the TELECOM System Technology
Award from the Telecommunications Advancement Foundation in 2004 and 2014, the Academic Encouragement Prize from IEICE in 2006,
the Itakura Prize Innovative Young ResearcherAward from ASJ in 2008, the Commendation for
Science and Technology by the Minister of Education, Culture, Sports, Science and Technology
Young Scientists’ Prize in 2014, an IEEE Best
Paper Award in 2015, and an IEEE ASRU 2015
Best Paper Award Honorable Mention in 2015.
She is a member of IEEE, IEICE, and ASJ.

Atsunori Ogawa

Senior Research Scientist, Signal Processing
Research Group, Media Information Laboratory,
NTT Communication Science Laboratories.
He received a B.E. and M.E. in informationengineering and a Ph.D. in information sciencefrom Nagoya University, Aichi, in 1996, 1998,
and 2008. He joined NTT in 1998. He is engaged
in research on speech recognition and speechenhancement. He is a member of IEEE and theInternational Speech Communication Association (ISCA), IEICE, the Information Processing
Society of Japan (IPSJ), and ASJ. He received
ASJ Best Poster Presentation Awards in 2003 and2006.

Keisuke Kinoshita

Senior Research Scientist, Signal Processing
Research Group, Media Information Laboratory,
NTT Communication Science Laboratories.
He received an M.Eng. and a Ph.D. from
Sophia University, Tokyo, in 2003 and 2010.
Since joining NTT in 2003, he has been researching speech and audio signal processing. His
research interests include single- and multichannel speech enhancement and robust automatic
speech recognition. He received a Paper Award
from the Institute of Electronics, Information andCommunication Engineers (IEICE) in 2006, ASJ
Technical Development Award in 2009, an ASJ
Awaya Young Researcher Award in 2009, a Japan
Audio Society Award in 2010, and the Maejima
Hisoka Award in 2017. He is a member of IEEE,
ASJ, and IEICE.

NTT Technical Review

Vol. 16 No. 11 Nov. 201823

Feature Articles

Tomohiro NakataniGroup Leader and Senior Distinguished
Researcher, Signal Processing Research Group,Media Information Laboratory, NTT Communication Science Laboratories.
He received a B.E., M.E., and Ph.D. from
Kyoto University in 1989, 1991, and 2002. Sincejoining NTT as a researcher in 1991, he has been
investigating speech enhancement technologies
for developing intelligent human-machine interfaces. He was a visiting scholar at Georgia Institute of Technology in 2005. Since 2008, he has
been a visiting assistant professor in the Department of Media Science, Nagoya University,
Aichi. He received the 2005 IEICE Best Paper
Award, the 2009 ASJ Technical Development
Award, the 2012 Japan Audio Society Award, the2015 IEEE ASRU Best Paper Award Honorable
Mention, and the 2017 Maejima Hisoka Award.
He was a member of the IEEE SP Society AASPTC from 2009 to 2014 and served as the chair of
the AASP-TC Review Subcommittee from 2013
to 2014. He has been a member of the IEEE SP
Society SL-TC since 2016. He served as an associate editor of the IEEE Transactions on Audio,
Speech and Language Processing from 2008 to
2010, Chair of the IEEE Kansai Section Technical Program Committee from 2011 to 2012,
Technical Program co-Chair of IEEE WASPAA-2007, Workshop co-Chair of the 2014
REVERB Challenge Workshop, and as a General
co-Chair of the IEEE ASRU. He is a member of
IEICE and ASJ.

NTT Technical Review

Vol. 16 No. 11 Nov. 201824

Feature Articles: Further Exploring Communication
Science

Network Reliability Optimization by
Using Binary Decision Diagrams
Masaaki Nishino, Takeru Inoue, Norihito Yasuda,
Shin-ichi Minato, and Masaaki Nagata
Abstract

We introduce an algorithm that can automatically identify communication network topologies that arerobust against failures. Robustness is usually assessed by the metric of network reliability. Since communication networks are a critical infrastructure, designing networks with high network reliability values is essential. However, the problem of finding a network topology that offers the maximum network
reliability is a computationally difficult problem, and previous methods therefore restrict their application area to very small networks. Our proposed method exploits the novel data structure called binary
decision diagrams, which makes it possible to find the most reliable network topology for communication networks with more than 10 times as many nodes (100) than is possible with previous methods.
Keywords: network reliability, optimization, binary decision diagram

1. Network reliability
Communication networks have become a key infrastructure and so must work without failure. However,network components such as links and nodes may failfor several reasons. Since these failures are inevitable, communication networks must be designed sothat they continue to function even if these failuresoccur. How can we design such networks?
An example of a simple communication network isshown in Fig. 1(a). Since the network connects two
terminals with one link, the network will fail if thelink fails. In contrast, a network with an additional
link is shown in Fig. 1(b). This network will continue
to work if one of the links fails. Therefore, this network is more reliable than the single-link network.
Network reliability is a measure used for quantifying how robust a communication network is against
failures. It is defined as the probability that the network will continue to support communications
assuming that the failure of its components follows
some probabilistic distributions.
Let us compute the network reliabilities of the networks in Fig. 1. We assume that each link fails independently with a probability of 20%. Since the probNTT Technical Review

Vol. 16 No. 11 Nov. 2018

ability of the network in Fig. 1(a) working equals theprobability that the link between terminals works, its
network reliability is 80%. In contrast, the network in
Fig. 1(b) will continue to work unless both links failsimultaneously. Since the probability of such an event
happening is 20% × 20% = 4%, the reliability of the
network is 96%. In this way, network reliability canquantify the robustness of a communication network.
We note, however, that evaluating network reliability
is a computationally difficult problem and becomes
infeasible for large networks.
2. Network reliability maximization
If a communication network is assessed to have sufficient reliability, we can continue to use it without
modification. If, however, the reliability is insufficient, remedial action is needed. A typical approachis adding links to the network since that always
improves the reliability. The task of finding the best
way to add links to improve reliability can be formulated and solved as the combinatorial optimization*1
problem called the network reliability maximization
problem. In what follows, we make the realistic
assumption that the total budget for adding links to25

Feature Articles

(a)

(b)

Fig. 1. Communication network examples. Network (b) is more robust against link failures than network (a).

Adding links to maximize network reliability while holding the total cost below 10
The costs and failure probabilities are listed in the table.

a

d
c

b

e

Link

Fail
probability

Cost

a

5%

2

b

8%

3

c

9%

3

d

7%

3

e

6%

1

Fig. 2. Budget constrained network reliability maximization problem.

the network is limited, and we want to maximize network reliability under this budget constraint. We callthis problem the budget constrained network reliability maximization problem.
The input of this optimization problem is a communication network that consists of nodes and links,
the total budget, and a set of candidate positions foradding new links. We assume that for each candidate
position, the cost for adding a link to the position, andthe probability that the added link will fail are known
in advance. The output of the problem is the communication network topology that achieves the maximum network reliability score and satisfies the constraint that the total cost incurred in adding links does
not exceed the budget. An example of a budget constrained network reliability maximization problem
and the set of candidate solutions of the problem arerespectively shown in Figs. 2 and 3. Of the candidate
solutions shown in Fig. 3, the center one is the optimal solution—the communication network topologythat achieves the maximum reliability among thosesatisfying the budget constraint.
We have seen that we can design a communication
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

network that is robust against failures by solving the
network reliability maximization problem. However,this problem is known to be computationally hard; ittakes a prohibitively long time even if we exploit
powerful modern computers.
A straightforward approach to solve the network
reliability maximization problem is to first enumerate
all candidate network topologies that can be madewhile satisfying the budget constraint and then evaluating the network reliability of each candidate. However, this simple approach has two potential problems. First, the number of candidate topologies satisfying a constraint may grow exponentially with network size. Second, evaluating the reliability of acandidate solution also takes an exponential amountof time. To evaluate the reliability of a network, wemust enumerate all the possible link failure patterns
with which the network works. Since the number ofsuch failure patterns grows exponentially with
*1 Combinatorial optimization: The problem of finding the best
combination from the set of combinations that satisfies given
constraints. Traveling salesman problems and knapsack problems
are typical examples of combinatorial optimization problems.

26

Feature Articles

Reliability

93%

Reliability

98%

Reliability

94%

Fig. 3. Example solutions for the problem in Fig. 2. The network in the center is an optimal solution.

adcompresscbe(a) Communication network

0(b) Failure patterns where
the network is still active(16 patterns)

1(c) 10-vertex BDD that
represents a set of 16 failure
patterns

Fig. 4. Using a BDD to represent failure patterns.

network size, the computation time also grows exponentially. Due to these two difficulties, previous
methods can find optimal solutions only for verysmall networks, that is, networks with fewer than 20
nodes.
3. Efficient optimization withbinary decision diagrams
We propose an efficient algorithm*2 for finding the
optimal solutions of budget constrained reliability
maximization problems [1]. This algorithm can findoptimal solutions for communication networks withmore than 100 nodes. It can handle networks that are
10 times the size of those possible with previous
methods. Moreover, the proposed method works
more than 10 thousand times faster than existing
methods. The key to our algorithm is that it uses thedata structure called binary decision diagrams(BDDs) [2].
BDDs can represent a set of failure patterns in a
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

compressed form. We show in Fig. 4 a communication network (Fig. 4(a)) and its possible failure patterns where the network still survives (Fig. 4(b)). In
Fig. 4(c), we show a BDD representing the set of
failure patterns in Fig. 4(b). A BDD is a directed
graph that consists of two types of vertices—the circles and rectangles in the figure. Every circle vertex
has two arcs—a solid arc and a dashed arc—and isassociated with a link of the communication network
in Fig. 4(a). Every rectangle vertex has a label of
either 1 or 0 and they are placed at the bottom.
In the BDD shown, every failure pattern in Fig. 4(b)corresponds to a directed path in the BDD from theroot BDD vertex to the rectangle terminal vertex withlabel 1. We can obtain the path that corresponds to a
failure pattern in the following way; given a failure
pattern, we first select the root BDD vertex (say v).
Then we check the link associated with the label of v.
*2 Algorithm: A computation procedure for solving a problem. A
computer can solve various problems by running algorithms.

27

Feature Articles

If the link works in the failure pattern, we follow thesolid arc and update v to the node reached. Otherwise,
if the corresponding link fails in the current failure
pattern, we follow the dashed arc of v and update v tothe node reached.
By repeatedly updating v depending on its labels,
we obtain a path in the BDD. For example, the failure
pattern marked by the green circle in Fig. 4(b) corresponds to the path on the BDD represented by thegreen arrow. By representing each failure pattern as apath, the BDD can share equivalent partial paths andthus represent the set of paths in a succinct way. For
example, the BDD in Fig. 4(c) represents 16 failure
patterns as a directed graph with 10 vertices. Since
we need 5 vertices for each failure pattern if they are
represented as paths, the compression ratio of thisBDD is 12.5%. When the input network is large, thecompression rate is smaller.
By using a BDD to compress the set of failure patterns, we can accelerate some of the computations
needed in solving budget constrained network reliability maximization problems. First, we can accelerate network reliability evaluation. By using a BDD,
we can precisely evaluate network reliability in time
linear to the number of BDD vertices. If the compression ratio of a BDD is 99%, just 1% of the originalcomputation cost is needed to evaluate network reliability. Second, a BDD can be used to estimate theamount of improvement that can be achieved by adding a link to the network. Such estimations allow us
to efficiently discard candidate network topologies
that may not achieve high reliability. This can also
reduce the computation cost needed to find an optimal solution. In this way, BDDs enable us to optimize

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

the reliability of large communication networks.
Up to this point we have focused on the problem ofmaximizing network reliability under budget constraints. In the real world, another design goal is toachieve reliability higher than a given threshold value
at minimum cost. Our algorithm can be applied to thisproblem and can efficiently find the minimum cost
solution.
4. Conclusion
We have introduced an efficient algorithm that canfind network topologies that maximize network reliability. This algorithm can be applied to designinginfrastructure networks in addition to communication
networks such as road networks, rail networks, andpower transmission networks, all of which demandhigh reliability. Of course, we have to consider several aspects other than reliability when designingcommunication networks. The most reliable network
might not be the best one if other aspects are considered. Future work includes enhancing our optimization method so that it can simultaneously handle
multiple aspects, one of which is reliability.
References
[1]

[2]

M. Nishino, T. Inoue, N. Yasuda, S. Minato, and M. Nagata, “Optimizing Network Reliability via Best-first Search over Decision Diagrams,” Proc. of the IEEE International Conference on Computer
Communications (INFOCOM 2018), pp. 1817–1825, Honolulu, HI,
USA, Apr. 2018.
R. E. Bryant, “Graph-based Algorithms for Boolean Function Manipulation,” IEEE Transactions on Computers, Vol. C-35, No. 8, pp.
677–691, 1986.

28

Feature Articles

Masaaki Nishino

Research Scientist, NTT Communication Science Laboratories.
He received a B.E., M.E., and Ph.D. in informatics from Kyoto University in 2006, 2008, and2014. He joined NTT in 2008. His current
research interests include data structures, natural
language processing, and combinatorial optimization.

Takeru Inoue

Senior Researcher, NTT Network Innovation
Laboratories.
He received a B.E., M.E., and Ph.D. from
Kyoto University in 1998, 2000, and 2006. He
was an ERATO (Exploratory Research for
Advanced Technology) researcher at the Japan
Science and Technology Agency (JST) from2011 through 2013. His research interests widely
cover the design and control of network systems.
He received the Best Paper Award from the AsiaPacific Conference on Communications in 2005
and research awards from the Institute of Electronics, Information and Communication Engineers (IEICE) Technical Committee on Information Networks in 2002, 2005, and 2012. He is amember of the Institute of Electrical and Electronics Engineers (IEEE).

Norihito Yasuda

Senior Researcher, NTT Communication Science Laboratories.
He received a B.A. in integrated human studies
and an M.A. in human and environmental studies
from Kyoto University in 1997 and 1999, and a
D.Eng. in computational intelligence and systemscience from Tokyo Institute of Technology in2011. He joined NTT in 1999. He also worked asa research associate professor with the Graduate
School of Information Science and Technology,
Hokkaido University, in 2015. His current
research interests include discrete algorithms andnatural language processing.

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Shin-ichi Minato

Professor, Graduate School of Informatics,
Kyoto University.
He received a B.E., M.E., and D.E. in information science from Kyoto University in 1988,
1990, and 1995. He worked in the NTT laboratories from 1990 until 2004. He was a visiting
scholar in the Computer Science Department at
Stanford University, USA, in 1997. He joined
Hokkaido University as an associate professor in2004 and was promoted to professor in October2010. He moved to Kyoto University in 2018(present position). He has also worked as a visiting professor at the National Institute of Informatics since 2015. He was a research director of
the JST ERATO Minato Discrete Structure
Manipulation System Project from 2009 to 2016.
His research interests include efficient representations and manipulation algorithms for largescale discrete structures such as Boolean functions, sets of combinations, sequences, and permutations. He is a senior member of IEICE andthe Information Processing Society of Japan(IPSJ) and a member of IEEE and the JapaneseSociety for Artificial Intelligence (JSAI).

Masaaki Nagata

Senior Distinguished Researcher, Group
Leader, NTT Communication Science Laboratories.
He received a B.E., M.E., and Ph.D. in information science from Kyoto University in 1985,
1987, and 1999. He joined NTT in 1987. His
research interests include morphological analysis, named entity recognition, parsing, andmachine translation. He is a member of IEICE,IPSJ, JSAI, the Association for Natural Language Processing, and the Association for Computational Linguistics.

29

Feature Articles: Further Exploring Communication
ScienceUkuzo—A Projection MappingTechnique to Give Illusory Depth
Impressions to Two-dimensional
Real Objects
Takahiro Kawabe
Abstract

Ukuzo is a light projection technique that gives illusory depth impressions to two-dimensional real
objects by projecting cast shadow images onto them. The technique can not only give depth impressions
but also manipulate material impressions of the object with the projected shadow patterns. The technique
is promising for enhancing the expression in paper-based advertisements and visual arts.
Keywords: projection mapping, visual illusion, augmented reality

1. Introduction
Painters use a variety of techniques to elaborately
express depth. For example, shadowing has been
employed as an important technique to increase therealistic impression of objects in paintings. In medieval Europe, it was prohibited to use shadowing as a
drawing technique in pictures for religious reasons.
In the Renaissance period, Leonardo De Vinci, aneminent painter as well as scientist, reintroduced
shadowing as a drawing technique and used it toenhance the realistic impression of his paintings [1].
Shadowing has also recently been used digitally in
the graphical user interface of computers. For example, it is used to give illusory depth impressions tovirtual objects shown on the computer display.
A vision science study [2] proposed an interestingillusion in which a shadow produces an illusory
three-dimensional (3D) layout of an object. This is
illustrated in the two pink squares shown in Fig. 1.
The right square is perceived as being more separate
from its background than the left one. In fact, the twosquares have identical image information. The difference between them is the spatial relationship between
the square and its shadow. The distance between the
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

right square and its shadow is larger than the distance
between the left one and its shadow. The previous
study reported that as the distance between an object
and a shadow increases, the object is perceived toapparently float up from its background to a greater
extent [2].
Another study [3] on a light projection technique
reported that light projection of a shadow motion cangive wobbly motion impressions to a miniature car,indicating that the projection of a cast shadow cangive illusory motion impressions to a real object.
We recently developed a light projection techniqueto give illusory depth perceptions that make it look as
if an object in a 2D image was floating upward. Thisis caused by perceptual effects coming from the projected pattern of a shadow. We call this technique
Ukuzo.
In this article, the hardware and software systems of
Ukuzo are explained, and perceptual experiences thatUkuzo offers to observers are described. The role oflight projection techniques in future applications ofspatial augmented reality is also explained.

30

Feature Articles

Fig. 1. Effect of a shadow on the depth of an object.

Video projector

Cameraprojection target. Finally, the created shadow patternis projected to the spatial vicinity of the projection
target so that it seems as if the projection target iscasting the shadow pattern.
3. Perceptual effects of Ukuzo

Computer
Projection
targetFig. 2. Photograph of Ukuzo hardware components.

2. Ukuzo system
A photograph of the Ukuzo hardware system isshown in Fig. 2. A shadow pattern is emitted onto a
projection target from a video projector. Since it is
impossible to project black light, the shadow pattern
we employed consists of a darker region serving as ashadow against a brighter background. A digital camera is used to capture the image of the projection target and to conduct geometric calibrations. We createa shadow pattern based on the captured image of the
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

The main visual effect of Ukuzo is to give illusory
depth impressions to 2D real objects. A photographof the Ukuzo effect is shown in Fig. 3. A dynamic
loop—the NTT logo—is printed on a piece of paper,and Ukuzo conveys a cast shadow pattern to the
dynamic loop. The viewer gets the impression thatthe dynamic loop is floating up from its original position. Moreover, the distance between the dynamic
loop and the cast shadow pattern can be dynamicallychanged. In this way, Ukuzo can give the illusory
perception that the dynamic loop is dynamicallychanging its depth positions. As the previous study[2] indicated, it is possible to change the apparent
depth position by manipulating the degree of blurringof a shadow pattern.
Ukuzo can give not only depth impressions but alsotransparency impressions to an opaque paper object.
A photograph of printed materials in which two diskshaped areas with different colors are overlapping in
the middle is shown in Fig. 4(a). In Fig. 4(b), Ukuzo
projects a shadow pattern to the greenish disk shapeon the left. When we view the effect shown in Fig. 4(b),the left disk appears to consist of a transparent greenish material such as glass or plastic. Similarly, in
31

Feature Articles

1. Normal printed material

3. Illusory depth perception

2. Cast shadow pattern

Projection

Input

Video projector

Fig. 3. A dynamic loop (NTT logo) with a cast shadow pattern projected by Ukuzo.

(a)

(b)

(c)

(d)

Fig. 4. (a) Photo of printed materials in which two disk shapes are overlapping. (b) The left disk has a cast shadow patternprojected by Ukuzo. (c) The right disk has a cast shadow pattern projected by Ukuzo. (d) The cast shadow patternprojected on the right disk shape.

Fig. 4(c), Ukuzo projects a shadow pattern to thereddish-colored disk on the right. This gives the effect
NTT Technical Review

Vol. 16 No. 11 Nov. 2018that the right disk shape is made of a transparent
material. The cast shadow pattern used in Fig. 4(c) is
32

Feature Articles

Fig. 5.

(Left) Handwritten words. (Right) Cast shadow patterns are projected onto the handwritten words.

shown by itself in Fig. 4(d). Such patterns can beprojected onto 2D printed material such as that shownin Fig. 4(a). When the shape of the projected castshadow pattern is elaborated, Ukuzo can give theimpression that opaque paper materials are made oftransparent materials such as glass or plastic.
Moreover, Ukuzo can give depth impressions to letters and illustrations that users draw. As shown in
Fig. 2, the Ukuzo system has a camera to capture a
projection target. The system captures the images ofhandwritten letters and illustrations, then generates
the cast shadow pattern based on the captured images.
When Ukuzo projects the cast shadow pattern onto
the handwritten letters and illustrations, it gives illusory depth impressions to the handwritten materials(Fig. 5).
4. Future of Ukuzo
Ukuzo can be categorized as a kind of spatial augmented reality technique that is generally designed
for changing the appearance of real objects by projecting digital images through a video projector. Spatial augmented reality techniques are usually intended
to change the appearance of real objects in a physically correct manner. Specifically, if engineers want
to change the appearance of a piece of paper from
opaque to transparent, they need to calculate andproject the desired light transport in order to display
the appearance of a transparent sheet. However, ingeneral, the light intensity of a typical projector is not
sufficiently strong to achieve the physically correct
light transport in ambient lighting.
In contrast, Ukuzo is not aimed at changing theappearance of real objects in a physically correct way.

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Rather, Ukuzo takes advantage of visual illusions
wherein observers recognize a projected dark region
as the shadow of an object and uses the shadow toestimate the 3D location of the object. Because it ispossible for a standard projector to project the darker
region in the vicinity of an object, it is feasible to use
Ukuzo under normal ambient lighting.
Ukuzo can be used in various scenes. For example,
Ukuzo can modify depth impressions of objects inpaper posters that are displayed in stores and transport stations. Paper posters cannot usually be edited
after printing. By making use of Ukuzo, designers
can highlight the portions of posters on which theywant customers to focus. Moreover, when a shadow
pattern is projected toward an area of text on theposter that might be overlooked, it is possible to getcustomers to notice it. Furthermore, Ukuzo may beable to motivate children to draw pictures on the basisof interactions between hand-drawn pictures and illusory depth impressions that Ukuzo can give to pictures. That is, Ukuzo may help to support children’s
artistic capabilities. Adults can also enjoy the effectsof Ukuzo when cast shadow patterns are conveyed totheir calligraphy work or woodcut prints. That is,
Ukuzo effects can be employed as a technique in
future visual arts.
References
[1]
[2]
[3]

F. Fiorani, “The Colors of Leonardo’s Shadows,” Leonardo, Vol. 41,
No. 3, pp. 271–278, 2008.
D. Kersten, D. C. Knill, P. Mamassian, and I. Bülthoff, “Illusory
Motion from Shadows,” Nature, Vol. 379, No. 6560, p. 31, 1996.
R. Raskar, R. Ziegler, and T. Willwacher, “Cartoon Dioramas in
Motion,” Proc. of the 2nd International Symposium on Non-Photorealistic Animation and Rendering (NPAR 2002), pp. 284–300, Annecy,
France, June 2002.

33

Feature Articles

Takahiro Kawabe

Senior Research Scientist, Sensory Representation Group, Human Information Science Laboratory, NTT Communication Science Laboratories.
He received a Doctor of Psychology from
Kyushu University, Fukuoka, in 2005. In 2011,
he joined NTT Communication Science Laboratories, where he studies applied aspects of humanperception. He received the 2013 JPA Award forInternational Contributions to Psychology:
Award for Distinguished Early and Middle
Career Contributions from the Japanese Psychological Association. In 2018, he was also awarded the Young Scientists’ Prize of the Commendation for Science and Technology by the Ministerof Education, Culture, Sports, Science and Technology. He is a member of the Vision Sciences
Society and the Vision Society of Japan.

NTT Technical Review

Vol. 16 No. 11 Nov. 201834

Feature Articles: Further Exploring Communication
Science

Cross-media Scene Analysis:
Estimating Objects’ Visuals Only
from Audio
Go Irie, Hirokazu Kameoka, Akisato Kimura,
Kaoru Hiramatsu, and Kunio Kashino
Abstract

Human beings can get a visual image of the surrounding environment from sounds they hear. Can wegive similar capabilities to computers? In this article, we introduce our recent efforts in cross-media
scene analysis applied to estimate the type, location, and visual shape of objects in a scene based only
on sound sources recorded with multiple microphones.
Keywords: cross media, scene analysis, deep learning

1. Introduction
The success of deep learning has completelychanged the framework of media processing and recognition. Deep learning has already delivered superhuman performance that can be applied to address
many problems in image and audio recognition. More
important is that although media processing technologies for different types of media (e.g., images,
video, audio, sound, and language) had at one time
mostly been studied and developed independently,
they are now being looked at together as their frameworks are similar.
In this article, we introduce cross-media scene
analysis technology that can predict image recognition results only from sound information. As thenumber of people who are interested in security/
safety increases day by day, the importance of surveillance and crime prevention technology is increasing. Most such technologies achieve excellent performance by leveraging advanced image recognition
techniques. However, such technologies are probably
not applicable in very dark places or rooms that have
many blind spots, or in private or public spaces where
privacy is prioritized and cameras are prohibited.
Our technology aims to provide visual recognition
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

functionality without using any camera devices. This
will make it possible to provide safe and comfortable
monitoring even in those places where normal image
recognition technologies cannot be used.
2. Cross-media scene analysis
Human beings recognize and understand their surrounding environments using their eyes and ears.
More interestingly, we integrate and use these signals
cross-sectionally to estimate the states of a scene. For
example, when we are walking on a road and suddenly catch the sound of a car coming from behind us,
we can infer how far away it is and possibly evenwhat type of car it is without actually turning aroundand looking at the car.
What we aim to do here is to equip a computer withthis kind of human ability. More specifically, thetechnology introduced in this article is designed to
predict an image recognition result as if it had been
photographed and recognized by a camera, using
only the sound recorded by microphones.
An example of a use case of our technology isshown in Fig. 1. Suppose there are two people in aroom and the objective is to automatically recognize
them using a computer. The initial idea would be to35

Feature Articles

Image recognition
Semantic
segmentation(a) Conventional image recognition to analyze bright room

Image recognition
Semantic
segmentation(b) Conventional image recognition to analyze dark room

Our technology
Cross-media scene
analysis(c) Our cross-media scene analysis to analyze dark room

Fig. 1. (a, b) Conventional image recognition and (c) our cross-media scene analysis technology.

install a camera in the room and recognize them byusing image recognition technology. If semantic segmentation technology is used, it is possible to detect
not only the presence or absence of people, but also
their locations and postures in silhouette form, as
shown in Fig. 1(a). However, this approach would notwork if the room was very dark or was a place where
cameras are prohibited (Fig. 1(b)).
However, our technology can still be applied insuch cases. Our technology uses audio information
recorded by multiple microphones to deliver semantic segmentation results, instead of using a camera(Fig. 1(c)). If two people in the room are talking, their
voices are recorded by the microphone arrays. With
this sound information, our technology directly predicts the expected semantic segmentation result. To
achieve this, we need to know (1) what kind of soundis coming from which direction, and (2) what kind of
object (and its shape) is generating the sound. These
details are respectively determined by signal processing and deep learning.
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Imagine that a sound occurs at some location and is
captured by multiple microphones. In this case,microphones closer to the source catch the soundearlier than the more distant ones. By analyzing thistime difference of arrival, we can extract a directionalfeature that indicates the direction of the soundsource. Furthermore, by analyzing the frequency
information of the sound captured by each microphone, we can obtain a tonal feature that is useful toidentify the type of the sound source (e.g., whether itsounds like a human voice or a train running on a
track). With these features, we can determine thedirection and the type of sound source. However, thisinformation is not enough to recover a visual silhouette of the sound source representing the position andshape as compared to the semantic segmentation
result.
Therefore, a deep neural network is used to estimate the type, shape, and position of the object. The
overall setup is illustrated in Fig. 2. The neural network receives directional and tonal features as inputs
36

Feature Articles

Microphones

Camera

Image
Sound

Signalprocessing

DirectionalfeatureSemantic
segmentation

TonalfeaturePredicted semantic
segmentation

Desirable semantic
segmentation result

Learning
Neural network

Fig. 2. Overview of cross-media scene analysis setup.

and is trained to output the semantic segmentation
result directly from them. This is the core part of ourtechnology that uses deep learning to convert media
types from audio to visual. This is why we call our
technology cross-media scene analysis. To train the
network, we need to have a pair consisting of a desirable semantic segmentation result and corresponding
audio features. Of course, a camera is needed to collect such data for training, but it is not necessary for
actual recognition. With this process, we can build abasic mechanism to predict semantic segmentation
results using only sound.
3. Proof of concept
We have conducted various experiments to evaluate
how well our technology can predict actual semantic
segmentation results from sound. We describe a few
examples here.
First, we introduce an experiment using simulation
data that was conducted to verify the feasibility ofthis technology in an ideal situation. We developed asimulator that can reconstruct a virtual room and generate conversational voices of people speaking andtheir corresponding semantic segmentation result
simultaneously (Fig. 3(a)). We can freely change the
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

size of the room, arrange people (three-dimensionalmodels), the virtual microphones and camera, andsimulate sounds recorded by the microphones taking
the reverberation and echo of the room into account.
At the same time, we can also simulate the image ofthe room taken by the camera and obtain the corresponding semantic segmentation result of the scene.
Hence, we can train our neural network and measure
how accurate the semantic segmentation result predicted by our technology is by comparing it to thedesirable one.
We show an example of the desirable semantic segmentation result and the result predicted with ourtechnology in Fig. 3(b) and (c), respectively.
Although the predicted result does not accurately
recover details of posture and shape, we can see thatthe distance and rough shape can be successfully predicted by our technology.
Next, we describe an experiment done on realsounds. The task was to estimate the position andorientation of a toy train based on its running noise.
Our evaluation setup is shown in Fig. 4(a). It consisted of a toy train running on a circumferentiallyconnected rail and four microphones connected to apersonal computer in which our technology was
installed. The entire setup was covered by a clear
37

Feature Articles

File
Add 2D mode
CameraFemaleMaleDesirable semantic segmentation result

3D model of humans

Image captured byvirtual camera
Speech signal

Play

Virtual microphones
and camera(a) Simulator(b) Desirable semantic segmentation result(c) Semantic segmentation result
predicted by our technology

3D: three-dimensional

Fig. 3. Our simulator and predicted semantic segmentation result obtained by our technology.

Microphones
Clear box

Toy train
Microphones(a) Experimental setup(b) Actual position and orientation (c) Semantic segmentation resultof the toy train
predicted by our technology

Fig. 4. Experimental setup and predicted results by our technology.

NTT Technical Review

Vol. 16 No. 11 Nov. 201838

Feature Articles

acrylic box. An example of the semantic segmentation result predicted by our technology is shown in
Fig. 4(c). The blue region represents the silhouette of
the train. Compared to the position and orientation of
the real train (Fig. 4(b)), we can see that the estimatedposition and orientation are roughly consistent withit, even though it is a little blurry. This shows that our
technology can be applied to sounds other thanhuman voices.
4. Future development
We will continue working to improve and demon-

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

strate our technology toward application to a morenatural space. To date, we have successfully verified
the technology in structured environments such as
those reconstructed by our simulator or the clear box.
However, we need to make our method more robust
in order to apply it in more realistic situations wheremore complex types of noise exist. The types ofobjects that can currently be recognized by our method are limited (people and toy trains), so we will work
to expand those and test for more diverse categories.
We will continue to improve our technology toachieve secure monitoring with arbitrary media information suitable for the situation.

39

Feature Articles

Go Irie

Senior Research Engineer, Media Information
Laboratory, NTT Communication Science Laboratories.
He received a B.E. and M.E. in systems engineering from Keio University, Kanagawa, in2004 and 2006, and a Ph.D. in information science and technology from the University ofTokyo in 2011. He joined NTT in 2006 and has
been studying multimedia content analysis,
indexing, and retrieval. During 2011–2012, he
was a visiting researcher at Columbia University,
NY, USA. He is a member of the Institute ofElectrical and Electronic Engineers (IEEE) andthe Institute of Electronics, Information andCommunication Engineers (IEICE).

Hirokazu Kameoka

Senior Research Scientist, Media Information
Laboratory, NTT Communication Science Laboratories.
He received a B.E., M.S., and Ph.D. from the
University of Tokyo in 2002, 2004, and 2007. He
is currently a Distinguished Researcher and a
Senior Research Scientist with NTT Communication Science Laboratories, and an Adjunct
Associate Professor with the National Institute ofInformatics. From 2011 to 2016, he was an
Adjunct Associate Professor with the Universityof Tokyo. His research interests include audio
and speech processing and machine learning. He
has been an Associate Editor for the IEEE/ACM
Transactions on Audio, Speech, and Language
Processing since 2015 and a member of the IEEE
Audio and Acoustic Signal Processing TechnicalCommittee since 2017.

Kaoru Hiramatsu

Senior Manager, NTT Geospace Corporation.*
He received a B.S. in electrical engineering andan M.S. in computer science from Keio University, Kanagawa, in 1994 and 1996, and a Ph.D. in
informatics from Kyoto University in 2002. He
joined NTT Communication Science Laboratories in 1996, where he worked on the Semantic
Web, sensor networks, and media search technology. From 2003 to 2004, he was a visiting
research scientist at the Maryland Information
and Network Dynamics Laboratory, Universityof Maryland, USA. He is a member of the Information Processing Society of Japan (IPSJ) andJSAI.
*He was a Senior Research Scientist, Supervisor, and Leader of Recognition Research Group,Media Information Laboratory, NTT Communication Science Laboratories until June 2018.

Kunio Kashino

Senior Distinguished Researcher, Head ofMedia Information Laboratory, NTT Communication Science Laboratories.
He received a Ph.D. from the University ofTokyo for his pioneering work on music scene
analysis in 1995. He is also an adjunct professor
at the Graduate School of Information Scienceand Technology, the University of Tokyo, and avisiting professor at the National Institute ofInformatics (NII). He has been working on audio
and video analysis, recognition, and search algorithms. He is a senior member of IEEE and
IEICE, and a member of ACM.

Akisato Kimura

Senior Research Scientist, Research Planning
Section, NTT Communication Science Laboratories.
He received a B.E., M.E., and D.E. in communications and integrated systems from Tokyo
Institute of Technology in 1998, 2000, and 2007.
He has been with NTT Communication Science
Laboratories since 2007. He is engaged in workon pattern recognition, machine learning, anddata mining. He is a board member of the Japanese Society for Artificial Intelligence (JSAI), a
senior member of IEEE and IEICE, and a member of the Association for Computing Machinery(ACM) SIGMM (Special Interest Group on Multimedia) and SIGKDD (Special Interest Groupon Knowledge Discovery and Data Mining).

NTT Technical Review

Vol. 16 No. 11 Nov. 201840

Feature Articles: Further Exploring Communication
Science

Measuring, Understanding, andCultivating Wellbeing in the Ageof Technology
Junji Watanabe, Yuuki Ooishi, Shiro Kumano,
Monica Perusquía-Hernández, Takashi G. Sato,
Aiko Murata, and Ryoko Mugitani
Abstract

The development of information and communication technology (ICT) has brought efficiency andconvenience to our daily lives. However, it has also been observed to have a negative impact on the emotional state of users. The concept of design for wellbeing is currently drawing a lot of attention. Thisapproach seeks to explore how ICT can support human psychological wellbeing. In this article, we
address several questions related to the issue of how to exploit technology and design to improve thewellbeing of humans.
Keywords: wellbeing, human science, self-tracking

1. Introduction
People can now access information anywhere andanytime as a result of the development of informationand communication technology (ICT). While the efficiency of intellectual tasks has improved dramaticallyas a result, there is now concern about the negative
impact of ICT on people’s mental state. For example,
when users constantly pay attention to smartphone
screens and alerts, they cannot relax their mind andbody because they are forced to be in a psychologicalstate of agitation. Also, because search engines present content that they surmise users want to see, thereis the possibility that users will not come into contact
with serendipitous information. In addition, social
issues such as the high costs arising from addiction tosocial online games and bullying in private communication groups have also emerged.
Due to these trends, a paradigm shift in design that
calls for technology to be not just efficient but also tocontribute to wellbeing (WB) is taking place.
Research on WB and experimental evidence thatNTT Technical Review

Vol. 16 No. 11 Nov. 2018serve as the foundation of this design are being
demanded [1]. In fact, WB is one of the critical Sustainable Development Goals (SDGs) for 2030 adopted by the United Nations in 2015. In the field ofarchitecture, the WELL Certification for buildings
has already been adopted. This is an environmental
certification system that evaluates architectural spaces for how well they incorporate WB. Information
technology (IT) companies such as Google practice
mindfulness, and business magazines such as Timepublish articles on mindfulness and WB. These trends
suggest that the issue of WB has reached a level ofgeneral awareness.
2. What is WB?
WB means good physical, psychological, andsocial conditions. However, its exact definition differs depending on the field and purpose. WB is
broadly divided into three types. The first is medical
WB, in which medically sound functioning of themind and body is seen as the foundation of our life.
41

Feature Articles

Medical WB can be measured in health checkups that
we receive regularly and in questionnaires on mental
health. The second is hedonic WB. This concept considers WB as the subjective emotion of happiness at
the moment. In general, when people are asked “Areyou happy?”, the answer tends to indicate a temporary feeling, which is hedonic WB. The third type iseudaimonic WB. Eudaimonic WB is defined as astate of being that can demonstrate the potential capabilities of mind and body, feeling significant, andleading an active life amid relationships with people
around oneself. It is also expressed as flourishing [2].
From the perspective of eudaimonic WB, because
achieving a vibrant state over a certain period of time
is more important than momentary pleasure, WB in
this definition is not necessarily limited to the ideathat positive feelings are good and negative feelings
are bad. For example, the process of accomplishingsomething may involve temporary hardship. However, if it brings about the feeling of accomplishment in
oneself, it can be included in the process of eudaimonic WB. Thus, as seen above, there are several
definitions of WB. Research on WB in recent years
has not been limited to hedonic WB; rather, understanding eudaimonic WB through multidimensional
factors such as subjective reports and physiologicalresponses over a certain period of time is increasingly
the objective.
Like weather and the economy, eudaimonic WB is
a construct that can be understood by identifying specific factors. In the case of weather, quantitativeevaluation of factors such as the temperature, level ofhumidity, barometric pressure, and wind speed candetermine whether the weather is good or bad. Evaluation of WB also requires identification of factors andmultidimensional evaluation. In general, factors that
increase eudaimonic WB can be divided into several
types (Fig. 1). Factors focusing on one’s present stateinclude engagement, mindfulness, and motivation.
Furthermore, factors indicating acknowledgement ofself-affirmation are also included. These include selfesteem, self-competence, and self-compassion. There
are also factors not contained within the individual,
but which involve other people for smooth interpersonal relationships. These include empathy and compassion. In addition, there are also factors that transcend the boundaries of the individual person, such as
contributing to society.
Although the achievement of all or some of thesefactors can contribute to WB, which factors bringabout an increase in WB depends on each person. WB
also differs depending on one’s life stage, such as
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

(Self-esteem)(Self-competence)(Self-compassion)(Engagement)(Mindfulness)(Motivation)(Compassion)(Empathy)

Fig. 1. Examples of factors of WB.

before and after marriage. Factors of WB in the workplace and in the home are also different. It is therefore
important to clarify the time period of the person andthe situation when thinking about how to increase
WB. It is not possible to create a panacea that benefits
all. Rather, it is important to be aware of one’s own
WB and autonomously choose options to improve
WB based on scientific evidence.
3. Measuring WB
In this section, we describe methodologies on how
to measure factors related to eudaimonic WB. However, because there is not yet a unified methodology
on what should be measured to evaluate eudaimonic
WB and how, we give examples of current measurable phenomena and describe their possibilities. Phenomena that have been determined to be measurable
factors of WB can be broadly divided into three categories. The first category consists of physiologicalresponses such as changes in heart rate, respiratory
rate, body temperature, and hormonal levels. Measuring these factors enables one to read the basic state ofa person such as whether his or her body is in a stateof nervousness or relaxation. Changes in the cardiac
cycle in particular can be used as indicators to evaluate the activities of the autonomic nervous system’s
sympathetic nervous system (nervousness) and parasympathetic nervous system (relaxation). Also, ahormone called cortisol, which can be extracted from42

Feature Articles

saliva and other bodily secretions, is used to measurestress.
The second category consists of behavioral information such as facial expressions, vocal intonation,and limb movements. When people are feeling positive, this is clear from their features such as the movements of particular facial expression muscles and theintonation of their voice.
The third category involves listening directly todescriptions. We can have a person express his or her
conditions (subjective report), or ask others of theirimpressions of the person when viewed externally(third-person evaluation). Because the subjective
report of the same event can differ if the person is
asked to report how he or she feels at that point intime or if he or she is asked to reflect back on it after
a certain period of time, care must be taken on how
emotions are measured temporally. The content of thereport may describe not only a person’s bodily healthand mental state, but also the subject’s own feelings
about the group or organization he or she belongs to.
WB in an organization is defined by replacing theindividual with the organization in the definition of
eudaimonic WB described above. It then refers to thestate of an organization where resources can demonstrate their potential capabilities, carry out sociallysignificant activities, and work actively in relationships with other organizations. It may be considered
to be independent from WB for individuals. If that isthe case, when we separate personal WB and organizational WB, it is possible to consider a balance of thetwo WBs and the factors contributing to this balance.
For eudaimonic WB, the factors need to be continually measured and comprehensively evaluated. Whatis critical is IT. At present, in addition to recording
one’s own subjective state, various technologies for
self-tracking such as recording body temperature,heart rate, blood pressure, and sleep, are being commercialized. Using IT for self-tracking makes it possible to gain an accurate understanding of one’s conditions based on data. Another advantage is the ability to compare one’s own state at a certain time to theenvironmental conditions at that time. A simple
example is the effect that a change in climate can haveon one’s state of health.

4. Research on WB at NTT Communication
Science Laboratories
Because WB is determined by interpersonal relationships at a variety of levels such as family, friends,
and organizations, it is necessary to analyze themechanisms at each level when researching WB. At
NTT Communication Science Laboratories, we haveinitiated research on WB and are measuring humanphysiological responses and elucidating empathy
mechanisms in communication. We are establishing afoundation to explore evidence-based WB mechanisms in human beings and study design guidelines
by integrating these research areas.
Specifically, in the area of human science, we areconducting research to scientifically understand theeffects of experiences such as listening to music andpracticing meditation, research on third-party evaluation of emotions to understand how a person’s emotions are recognized by others, research on understanding the principles of psychological and physicalbonds between mothers and their children, researchon the effects that being together with others have on
emotional response, and research on quantifying
crowd sensations such as at an event or stadium bymeasuring group physiological responses simultaneously. Furthermore, we are conducting research on
physiological response-linked tactile presentation
technologies and research on intervention.
In this way, by engaging in WB research from acomprehensive and interdisciplinary perspective, weare pursuing the essence of WB. At the same time, weare engaged in design methodologies to realize conditions in which people and organizations can sustain
vibrant activities (that is, WB). Specifically, we are
creating indicators and developing measurement
technologies for quantifying WB, suggesting productdesigns, service designs, and organizational designs,
and studying how to contribute to sustaining interpersonal relationships (child-rearing and relationships inlocal communities) in a new technological environment.
References[1][2]

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

R. Calvo and D. Peters, “Positive Computing - Technology for Wellbeing and Human Potential,” MIT Press, Cambridge, USA, 2014.
M. E. P. Seligman, “Flourish - A Visionary New Understanding ofHappiness and Well-being,” Atria Books, New York, USA, 2012.

43

Feature Articles

Junji Watanabe

Senior Research Scientist (Distinguished
Researcher), Sensory Resonance ResearchGroup, Human Information Science Laboratory,
NTT Communication Science Laboratories.
He received a Ph.D. in information science andtechnology from the University of Tokyo in
2005. He presented his work at technology showcases, science museums, and art festivals.

Yuuki Ooishi

Senior Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
He received a B.S., M.S., and Ph.D. in physics
from the University of Tokyo in 2003, 2005, and2008. His research interest is the mechanism ofneural dynamics, including auditory-evokedemotion, and autonomic nerve-neuroendocrinological interaction. He is a member of the Japan
Neuroscience Society.

Shiro Kumano

Senior Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
He received a Ph.D. in information science andtechnology from the University of Tokyo in 2009
and joined NTT Communication Science Laboratories the same year. His research interests
include affective computing, cognitive scienceand computer vision. He received the ACCV
(Asian Conference on Computer Vision) 2007
Honorable Mention Award. He serves as an organizing committee member of International Conference on Affective Computing and Intelligent
Interaction (ACII). He is a member of the Institute of Electrical and Electronics Engineers(IEEE) and the Institute of Electronics, Information and Communication Engineers (IEICE).

Takashi G. Sato

Senior Research Scientist, Moriya ResearchLaboratory, NTT Communication Science Laboratories.
He received a B.S., M.S., and Ph.D. in information science and technology from the Universityof Tokyo in 2003, 2005, and 2008. His researchinterests include human interfaces, especiallybrain-machine, audio, and tactile interfaces using
psychophysiological measurements as feedback
information. He is a member of IEEE, IEICE, theAcoustical Society of Japan, the Japan Neuroscience Society, and the Society of Instrument andControl Engineers.

Aiko Murata

Research Associate, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
She received a Ph.D. in social psychology from
Hokkaido University in 2016. She joined NTT
Communication Science Labs in 2018. Her
research interests include investigating themechanisms of human empathy.

Ryoko Mugitani

Senior Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
She received a B.A. in education in 1999, an
M.A. in health science in 2001, and a Ph.D. inarts and sciences in 2004 from the University ofTokyo. She joined NTT Communication Science
Labs in 2004. Her interests include developmentof communication and child wellbeing.

Monica Perusquía-Hernández

Research Associate, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
She received a B.Sc. in electronic systems
engineering from Instituto Tecnológico y deEstudios Superiores de Monterrey in 2009, andan M.Sc. in human–technology interaction and a
Professional Doctorate in engineering in user–
system interaction from the Eindhoven University of Technology, Eindhoven, The Netherlands,
in 2012 and 2014. Her research interests include
affective computing and biosignal processing.

NTT Technical Review

Vol. 16 No. 11 Nov. 201844

Regular Articles

High-speed Avalanche Photodiodes
toward 100-Gbit/s per Lambda Era
Masahiro Nada, Toshihide Yoshimatsu,
Fumito Nakajima, Hideaki Matsuzaki,
and Kimikazu Sano
Abstract

We have developed an avalanche photodiode (APD) in an effort to achieve 100-Gbit/s operation witha single wavelength (100-Gbit/s/lambda). The APD features a gap-grading layer and was set between theabsorber and avalanche layers. It achieved an operating bandwidth of 42 GHz in low-gain conditions. An
optical receiver made with the APD demonstrated 106-Gbit/s 4-level pulse amplitude modulation
operation and 40-km optical amplifier-free transmission over a single-mode fiber under assumption of a
KP4 forward error correction threshold.
Keywords: avalanche photodiode (APD), 100-Gbit/s PAM4, datacenter, optical receiver

1. Introduction
The interest in optical fiber communications systems in the last few decades has primarily been
focused on long-haul systems capable of handlingtranscontinental communications and country-tocountry communications. Such long-haul systems
have included advanced optical devices used for optical communications as well as advanced modulation
formats and detection mechanisms. One important
example is digital coherent systems [1]. Coherent
detection techniques used in combination with digital
signal processors have achieved transmission distances of several thousand kilometers, and the development of densely integrated optical transmitters andreceivers has enabled digital coherent systems toreach a transmission capacity of over 100 Gbit/s.
Recent interest in optical fiber communications
systems has been focused on short-reach applications
as represented by inter/intra-datacenter networking.
In 2010, 100-Gbit/s Ethernet, which utilizes a bit rate
per wavelength of 25 Gbit/s, was standardized forsuch purposes [2], and the required bit rate per wavelength is now approaching 100 Gbit/s (100-Gbit/s/
lambda). Unlike digital coherent systems, such shortNTT Technical Review

Vol. 16 No. 11 Nov. 2018

reach networks are required to be low-cost and thus
simple in structure. Consequently, the network systems and optical components used in short-reach
networks must achieve larger capacity and longer
transmission distances by using simple direct detection mechanisms and high-speed optical components.
An avalanche photodiode (APD) is a key device tomeet such requirements. Avalanche multiplication is
initiated by electrons and holes in the APD, which
provides a built-in first stage of gain of the electricalsignals. Therefore, APDs obtain higher responsivity
compared with the conventional PIN-PDs (positiveintrinsic-negative photodiodes), leading to higher
sensitivity of the optical receivers. Thus, they effectively extend the transmission distance. To date, wehave demonstrated 25-Gbit/s operation for NRZ(non-return-to zero) signals and 50-Gbit/s operationfor 4-level pulse amplitude modulation (PAM4) signals by using the high-speed APDs we developed[3–5].
These high-speed APDs surpass the results of ourprevious examples and are aimed at achieving
100-Gbit/s/lambda operations. A gap-grading layer
introduced between the absorber and the avalanche
layers helped to boost the operation speed of the45

Regular Articles

(b)

(a)

(c)

Field control

electronAbsorber
Avalanche

Electric fieldhole

V  VonGain: low

V > VonGain: medium

V >> Von
Gain: large

V: bias voltage
Von: onset voltage

Fig. 1. Schematics of band diagrams of APD for various bias conditions.

APD. An optical receiver fabricated with the APD
achieved 106-Gbit/s PAM4 transmission over a
40-km single-mode fiber without using an opticalamplifier.
2. Device design and performance
Low-gain operation must be considered when trying to boost the speed of APDs. In principle, there isa strict trade-off between the gain and the speed of the
APD according to the gain-bandwidth product (GBP)
rule that is governed by the material and thickness of
the avalanche layer; that is, the operation speed
decreases as the gain increases. Thus, one simple way
to obtain high-speed operation is to operate the APDwith lower gain. We employ a 90-nm-thick indium
aluminium arsenide (InAlAs) avalanche layer, which
provides larger GBP as well as low excess noise, in
our APD. However, one problem is that a large conduction band offset exists between the indium gallium arsenide (InGaAs) absorber and the InAlAs avalanche layer. This conduction band offset is as large
as 0.5 eV, which can be problematic for low-gain andhigh-speed operation. A schematic band diagram
illustrating this issue is shown in Fig. 1.
In Fig. 1(a), we show the band diagram around theabsorber and the avalanche layer when the bias voltage (V) is lower than the onset voltage (Von). In thisNTT Technical Review

Vol. 16 No. 11 Nov. 2018

condition, the electric field in the absorber is zero,and the potential barrier caused by the conduction
band offset remains. The photo-generated electrons
and holes do not have drift components; thus, highspeed operation of the APD is not observed in thiscondition. Additional bias voltage depletes the p-typefield control layer, and the electric field in the
undoped absorber is invoked, as shown in Fig. 1(b).
The photo-generated electrons and holes have drift
components in the absorber. However, the remaining
potential barrier originating from the conduction
band offset prevents electrons from moving toward
the avalanche layer. Consequently, high-speed operation cannot be obtained even under the V > Von condition depicted in Fig. 1(b).
To eliminate the potential barrier caused by the
conduction band offset, further additional bias voltage is needed (Fig. 1(c)). The additional bias voltage
effectively eliminates the potential barrier but simultaneously strengthens the electric field in the avalanche layer, resulting in the increased gain of the
APD. Consequently, the speed of the APD is loweredby the GBP limitation.
A gap-grading layer is known to effectively relax
the potential barrier [6]. We introduced a 40-nm-thick
1.1-eV indium aluminium gallium arsenide (InAlGaAs) gap-grading layer between the InGaAs
absorber and InAlAs avalanche layers to relax the46

Regular Articles

MirrorN-type contact
Edge-field buffer
N-type field control
Avalanche
P-type field control
Gap grading
Hybrid absorber
P-type contact
S.I. InP substrate
Anti-reflection coating
Signal light
S.I.: semi-insulating
InP: indium phosphide

Fig. 2. Schematic cross-sectional view of the fabricated APD.

potential barrier that prevents the electron transport
from the absorber to the avalanche layer under lower
bias conditions, which can boost the speed of theAPD.
A cross-sectional view of the APD we developedfor 100-Gbit/s/lambda (100-Gbit/s PAM4) operationis shown in Fig. 2. The APD is designed based on an
inverted p-down structure [7]. The epitaxial layers,
including the p-type contact, p-type absorber,undoped absorber, gap-grading, p-type field control,
avalanche, n-type field control, edge-field buffer, andn-type contact layers are grown on a semi-insulating
InP (indium phosphide) substrate by metal-organic
chemical vapor deposition. After the epitaxial growth,
we formed the triple-mesa structure shown in Fig. 2
using a wet-etching technique. Then we formed theelectrodes and mirror metal by electron beam evaporation. The anti-reflection film is coated after thefront-end process. Thus, the APD has a back-side
illumination structure.
As described above, we employed a 90-nm-thick
InAlAs avalanche layer. We also used a hybrid
absorber consisting of 150-nm p-type and 150-nm
undoped InGaAs to reduce the transit time of theelectrons and holes [8].
The current-voltage (I-V) characteristics of the fabricated APD with an active diameter of 14 μm areshown in Fig. 3. The photocurrent rapidly rises fromNTT Technical Review

Vol. 16 No. 11 Nov. 2018

5.5 V to 7.5 V and then increases gradually toward a
breakdown voltage (Vb) of 25.4 V due to the avalanche gain. The dark current does not show any
unexpected increase, indicating that the APD has noedge breakdown or other unexpected breakdown. The
estimated responsivity at unit gain is 0.5 A/W against
1300-nm-wavelength optical input.
The frequency characteristics of the fabricated
APD are shown in Fig. 4. A maximum bandwidth of
42 GHz was observed with a gain as low as 1.5 thanks
to the gap-grading layer in the APD. A bandwidth ofover 30 GHz, which is sufficient for 100-Gbit/s
PAM4 operation, was maintained for the increase ingain to 3.
3. Receiver characteristics
To demonstrate the 100-Gbit/s PAM4 operation of
the APD, we mounted the fabricated APD into a butterfly package together with a transimpedance amplifier and conducted a transmission test with the fabricated APD receiver. The transimpedance amplifier
had a bandwidth of 33 GHz, which is sufficient for
100-Gbit/s PAM4 operation. The package has a
GPPO electrical output. The optical transmitter weused consisted of a 1309.49-nm-wavelength electroabsorption modulator integrated DFB (distributed
feedback) laser (EML) with a launch power of +3.5
47

Regular Articles

1E-03
: 14 µm

1E-04
1E-05

Current (A)

1E-06
1E-07
1E-08
1E-09
Photocurrent

1E-10

Dark current

1E-11
1E-12
0

5

10

15

20

25

30

Bias voltage (V)Fig. 3. I-V characteristics of the fabricated APD with an active diameter of 14 μm.

3Optical/electrical response (dB)

0
−3
−6
−9
−12
−15
−18
−21
M = 1.5

−24

M=3

−27

M = 3.5

−30
0

10

20

30

40

50

Frequency (GHz)

Fig. 4. Frequency characteristics of the fabricated APD.

dBm [9]. The bit rate of the electrical signal thatdrives the EML was set to be 106-Gbit/s PAM4 byconsidering the overhead of the forward error correction (FEC). The 106-Gbit/s PAM4, PRBS (pseudorandom binary sequence) signal of 215-1 was generated with a 3-bit DAC (digital-to-analog converter).
The optical output signal from the EML had an
extinction ratio at 106-Gbit/s PAM4 operation of
about 7 dB. The electrical output signal was captured
NTT Technical Review

Vol. 16 No. 11 Nov. 2018by a real-time DSO (digital storage oscilloscope)with a 160-GS/s sampling rate and 62-GHz bandwidth.
The transmission tests were performed with offline
processing using feed-forward equalization (FFE)with a half-symbol-spaced (T/2-spaced) adaptive
equalizer. No pre-emphasis on the transmitter wasused. The received power was determined by the
VOA (variable optical attenuator) set in front of thereceiver. For the 40-km transmission test, we usedsingle-mode fiber (SMF) with a loss of 0.33 dB/km.
The measurement setup for the characterization of
the APD optical receiver is shown in Fig. 5.
The bit error rate (BER) characteristics of the fabricated APD receiver under back-to-back conditions
and after 40-km transmission are presented inFig. 6(a). The number of taps of the FFE was set to
be 17 for both conditions. In Fig. 6(b) and (c), we
respectively show the equalized electrical eye diagrams of the APD receiver for back-to-back and after
40-km transmission conditions. The BER characteristics in Fig. 6(a) indicate that the fabricated APD
receiver successfully demonstrated 106-Gbit/s PAM4
operation even for the 40-km transmission over SMF
without an optical amplifier, as well as in the back-toback condition. The 40-km transmission wasachieved with an average received power (Pavg.) of–11.47 dBm at a KP4-FEC limit. The power penalty
against the back-to-back condition was 0.32 dB. Theeye diagram obtained after 40-km transmission48

Regular Articles

106-Gbit/s PAM4
3-bit
DAC

PPG

EML

Back-to-back/40-km SMF
APD
receiverHard decision

BER counting

FFE (DD-LMS)

Resampling

DSO

VOA

Offline processing
BER: bit error rate
DD-LMS: decision-directed least mean square
PPG: pulse-pattern generator

Fig. 5. Experimental setup for 100-Gbit/s PAM4 bit error rate test utilizing APD receiver.

(b)

(a)
10−1
17-tap T/2-spaced FFE, after 40 km

BER

17-tap T/2-spaced FFE, back-to-back

10−2

(c)

10−3
KP4 FEC limit
10

−4

10−5
10−6
−20

−15

−10

−5

0

5

Average received power, Pavg. (dBm)

Fig. 6. (a) BER characteristics of the fabricated APD receiver for the back-to-back condition and after 40-km transmission.
The T/2-FFE is set to 17 taps. (b) Electrical eye diagram of the APD receiver at back-to-back condition and (c) after
40-km transmission.

NTT Technical Review

Vol. 16 No. 11 Nov. 201849

Regular Articles

(b)

(a)
10−1
17-tap T/2-spaced FFE, after 40 km

BER

7-tap T/2-spaced FFE, after 40 km

10−2

(c)

10−3
KP4 FEC limit
10

−4

10−5
10−6
−20

−15

−10

−5

0

5

Average received power, Pavg. (dBm)

Fig. 7. (a) BER characteristics of the fabricated APD receiver after 40-km transmission for 17-tap and 7-tap conditions, (b)electrical eye diagram of the APD receiver after 40-km transmission with 17 taps, and (c) with 7 taps.

shows no degradation from that of the back-to-backcondition.
A smaller number of FFE taps is advantageous interms of power consumption and latency by signal
processing when considering the digital-signal processing used in practical optical transceivers. The
BER characteristics of the APD receiver for 17-tap
and 7-tap FFE after 40-km transmission are shown in
Fig. 7(a). A minimum receiver sensitivity as low as–10.57 dBm was obtained, even with the 7-tap FFE.
The equalized eye diagram of the APD receiver with
7-tap FFE is presented in Fig. 7(b). No significant
degradation from the 17-tap condition is apparent,
which indicates that the 7-tap FFE is also applicablefor 40-km transmission without an optical amplifier
using the fabricated APD.

operation. Thanks to the gap-grading layer, the APD
exhibited a peak bandwidth of 42 GHz with a responsivity at unit gain of 0.5 A/W. A 30-GHz bandwidth
was maintained for a gain of 3. The optical receiverfabricated with the APD successfully achieved 40-km
optical-amplifier-free transmission for 106-Gbit/s
PAM4 optical signals. These results suggest that the
100-Gbit/s/lambda era can be realized even when
extending the transmission distance by using lowpower-consumption, cost-effective optical components that high-speed APDs provide.
References[1][2]

4. Conclusion
We developed a high-speed APD designed for use
in the 100-Gbit/s per wavelength era. We introduced
an optimized gap-grading layer consisting of 1.1-eV
InAlGaAs in order to achieve low-gain, high-speed
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

[3]

[4]

K. Kikuchi, “Digital Coherent Optical Communication Systems:
Fundamentals and Future Prospects,” IEICE Electronics Express, Vol.
8, No. 20, pp. 1642–1662, 2011.
IEEE P802.3ba 100 Gb/s Ethernet Task Force, http://www.ieee802.
org/3/ba/T. Yoshimatsu, M. Nada, M. Oguma, H. Yokoyama, T. Ohno, Y. Doi,
I. Ogawa, and E. Yoshida, “Compact and High-sensitivity 100-Gb/s (4× 25 Gb/s) APD-ROSA with a LAN-WDM PLC Demultiplexer,”
Proc. of the 38th European Conference on Optical Communication(ECOC 2012), Th.3.B.5, Amsterdam, The Netherlands, Sept. 2012.
F. Nakajima, M. Nada, and T. Yoshimastu, “High-speed Avalanche
Photodiode and High-sensitivity Receiver Optical Subassembly for50

Regular Articles

[5]

[6]
[7]

100-Gb/s Ethernet,” J. Lightw. Technol., Vol. 34, No. 2, pp. 243–248,
2016.
Y. Nakanishi, T. Ohno, T. Yoshimatsu, Y. Doi, F. Nakajima, Y.
Muramoto, and H. Sanjoh, “4 x 28 Gbaud PAM4 Integrated ROSA
with High-sensitivity APD,” Proc. of the 20th Opto-Electronics andCommunications Conference (OECC 2015), post-deadline paper,
Shanghai, China, June/July 2015.
J. C. Campbell, “Recent Advances in Telecommunications Avalanche
Photodiodes,” J. Lightw. Technol., Vol. 25, No. 1, pp. 109–121, Jan.
2007.
M. Nada, Y. Muramoto, H. Yokoyama, T. Ishibashi, and H. Matsuzaki,“Triple-mesa Avalanche Photodiode with Inverted P-Down Structure

Masahiro Nada

Research Engineer, NTT Device Innovation
Center.
He received an M.E. in applied physics fromthe University of Electro-Communications,Tokyo, in 2009 and a Ph.D. in engineering fromthe University of Tokyo in 2017. He joined NTT
Photonics Laboratories in 2009 and has since
been researching and developing high-speed,
high-responsivity APDs and their applications in
optical fiber communications systems. Dr. Nada
received the Young Researcher’s Award from theInstitute of Electronics, Information and Communication Engineers (IEICE) in 2014. He is amember of IEICE, the Optical Society (OSA),
and the Institute of Electrical and Electronics
Engineers (IEEE).

Toshihide Yoshimatsu

Senior Research Engineer, NTT Device Innovation Center.
He received a B.E. and M.E. in applied physics
from Tohoku University, Miyagi, in 1998 and2000. He joined NTT Photonics Laboratories in
2000. He is involved in researching and developing ultrafast opto-electronic devices. He received
the International Conference on Solid State
Devices and Materials (SSDM) Paper Award in
2004. He is a member of IEICE and the Japan
Society of Applied Physics (JSAP).

Fumito Nakajima

Senior Research Engineer, NTT Device Technology Laboratories.
He received a B.E., M.E., and Ph.D. from Hokkaido University in 1998, 2000, and 2003. He
joined NTT Photonics Laboratories in 2003. He
is involved in the research and development(R&D) of high-speed photodiodes and APDs for
optical receivers. He received the Young
Researcher’s Award in 2006 and the Achievement Award in 2018 from IEICE. Dr. Nakajima isa member of IEICE.

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

[8]

[9]

for Reliability and Stability,” J. Lightw. Technol., Vol. 32, No. 8, pp.
1543–1548, Apr. 2014.
M. Nada, T. Yoshimatsu, Y. Muramoto, H. Yokoyama, and H.
Matsuzaki, “Design and Performance of High-speed Avalanche
Photodiodes for 100-Gb/s Systems and Beyond,” J. Lightw. Technol.,Vol. 33, No. 5, pp. 984–990, 2015.
S. Kanazawa, S. Tsunashima, Y. Nakanishi, Y. Muramoto, H.
Yamazaki, Y. Ueda, W. Kobayashi, H. Ishii, and H. Sanjoh,“Equalizer-free 2-km SMF Transmission of 106-Gbit/s 4-PAM Signal
Using Optical Transmitter/Receiver with 50 GHz Bandwidth,” Proc.
of the 21st Opto-Electronics and Communications Conference(OECC 2016), ThD1-2, Niigata, Japan, July 2016.

Hideaki Matsuzaki

Senior Research Engineer, Supervisor, NTT
Device Technology Laboratories.
He received a B.S. and M.S. in physics from
Kyoto University in 1993 and 1995. He joined
NTT’s Atsugi Electrical Communications Laboratories in 1995. He is currently engaged in R&D
of compound semiconductor devices, photodiodes, and laser diodes. He is a member of IEICE.

Kimikazu Sano

Senior Research Engineer, Supervisor, NTT
Device Innovation Center.
He received a B.S., M.S., and Ph.D. in electrical engineering from Waseda University, Tokyo,
in 1994, 1996, and 2004. He joined NTT in 1996.
Since then, he has been designing and evaluating
ultrafast integrated circuits (ICs) and optoelectronic ICs. From 2005 to 2006, he was a visiting
researcher at the University of California, Los
Angeles (UCLA), USA, where he researched amicrowave/millimeter-wave sensing system. He
was with NTT Electronics from 2012 to 2014,
where he developed high-speed analog ICs andpackaged modules for coherent optical systems.
He is currently developing lasers, photodiodes,
and analog ICs for optical metro-access networks. He served as a member of the Technical
Program Committee for the IEEE Compound
Semiconductor IC Symposium (CSICS) from2008 to 2010.

51

Global Standardization Activities

Telecom Infra Project—Its Structure
and Activities
Takeshi Kinoshita, Hitoshi Masutani,
and Katsuhiro Shimano
Abstract

Telecom Infra Project (TIP) was launched in February 2016 through the initiative of Facebook. It isaimed at bringing about innovations in telecom hardware, software, and operations by introducing suchtechnical trends as openness and disaggregation, which have become increasingly important in recent
years. TIP is similar to open source projects in that communities play an essential role.
Keywords: TIP, openness, disaggregation

1. Background

2. Overview of TIP

In the technological fields involving datacentersand datacenter interconnections, de facto standards
are widely used. They come from products and specifications developed by organizations such as theOpen Networking Foundation as well as open source
projects. What is characteristic is that these standards
are developed through making the designs and interfaces open. In many cases, the resulting products arealso open source. The trend of openness is now seen
in not only the development of software but also of
hardware. For example, the Open Compute Project(OCP) pushed through openness in the specificationsof servers and racks used in datacenters, leading to a
reduction in the purchasing cost as well as in the
power consumption. In this approach, functions andcomponents were unbundled. It is this disaggregationthat made it possible for users to choose only the parts
necessary to them, eliminating others.
The trend of openness and disaggregation is now
beginning to expand to telecom networks. Telecom
Infra Project (TIP) was established with the leadership of Facebook, one of the leading companies of
OCP, and with the idea that the same approach as
OCP’s could be applied in telecom networks.

Since TIP started its activities in February 2016, thenumber of member companies has increased fromseveral dozen to over 500. NTT joined TIP in 2017.
The Board of Directors includes people from telecom
operators such as Deutsche Telekom (DT) and SK
Telecom, which have held the position since the foundation of the organization, and from Telefónica,
Vodafone, and British Telecom (BT). This demonstrates that European operators are actively involvedin the activities. Many operators from other regions
such as India, South Africa, and Brazil are also participating. For these operators whose service areas
include places where communications infrastructure
has not been well established, the open-sourced products and the know-how arising from TIP would serve
as a solution to address the issue adequately and costeffectively.
Manufacturers of white box equipment, which use
merchant chips and non-proprietary operating systems, and chip vendors are among those who have
expanded their presence in concert with the marketsof these products. There are also other kinds of participants such as Internet service providers and system integrators, including startup companies. Thereis a great deal of interest in the latest technical trends
and expectations for their implementation among thewide variety of participants.

NTT Technical Review

Vol. 16 No. 11 Nov. 201852

Global Standardization Activities

The organizational structure of TIP is shown in
Fig. 1. Project Groups (PGs), responsible for creating
specifications or developing products, are establishedafter their charters are accepted by the Board of
Directors and the Technical Committee. Althoughsome PGs have been dissolved, the number of PGs
has increased since TIP was founded, showing flexibility similar to that of other organizations in the fieldof datacenter technologies.
PGs are classified by their scopes into three technical fields: Access Projects, Backhaul Projects, andCore and Management Projects. The scopes of individual PGs are written in their charters along withgoals and deliverables. Products developed by a PG
are usually tested in a Community Lab. There are currently six Community Labs, two in the USA and one
each in Germany, South Korea, Brazil, and India.
Each Community Lab is hosted and operated by aspecific company with a policy of allowing the participation of member companies. As is evident inmany successful open source projects, creating open
communities is recognized as a key to advancing
development.
TIP Ecosystem Acceleration Centers (TEACs) provide startup companies with an environment for product development and tests. Currently, BT, DT,
Orange, and SK Telecom each host a TEAC.
3. Access Projects
The activities of the PGs whose scope is related to
the Access field are listed in Table 1 [1]. These PGs
aim to make Internet access easier by innovating
access network technologies. Reducing infrastructure
costs and creating new technologies that are applicable where existing ones cannot be easily used areamong the examples of the PGs’ goals.
The OpenCellular PG is working to produce basestations of Long-Term Evolution (LTE) cellular networks. While open source software is employed forthe management of the base stations, chips and otherhardware are developed by the participant companies, and their design has been made public as open
source hardware. This PG is focusing on the existing
mobile technology, rather than the emerging fifthgeneration (5G) mobile technology, and it hasbrought openness into its products, representing the
TIP’s concept. The products have been used in fieldtrials in India, Pakistan, and other African and SouthAmerican countries.
The activities of many of the other Access PGs arealso related to mobile communications infrastructure.
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Telecom Infra Project (TIP)– Board of Directors– Technical Committee– Project Groups (PGs)

Access / Backhaul /Core and Management– Community Labs
– TIP Ecosystem Acceleration Centers (TEACs)

Fig. 1. Organizational structure of TIP.

The vRAN Fronthaul PG is tackling issues on themobile fronthaul, which is responsible for the connection between antennas and baseband signal processing units of base stations. In developing countries, optical fibers cannot always be used as transmission media due to costs or environmental problems. The PG is addressing this issue by studying
how to implement fronthaul transmission using
twisted pair cables, coaxial cables, or other non-ideal
media.
A power supply can also be an impediment in
deploying infrastructure that costs or environmental
problems impose. The Power and Connectivity PG
aims to establish technologies and business models to
deploy power supply infrastructure in rural areas withlower costs, inviting power companies and battery
makers as members.
The OpenRAN PG’s scope is similar to that of the
OpenCellular PG’s in that both are aimed at developing LTE base stations. Their focus, however, is on
software implementation and its programmability.
Because the functionalities are programmed to workwith general purpose processors, flexible and timely
functional updates are possible.
4. Backhaul Projects
A rapid increase in the amount of communications
traffic is being seen in many countries and is a common issue for telecom operators worldwide. Against
this background, PGs related to the Backhaul field aretrying to innovate technologies in trunk and backbone
networks. The activities of the PGs focusing on thisfield are summarized in Table 2 [1].
The Open Optical Packet Transport PG is targeting
innovative optical fiber transmission techniques.
They have released two equipment prototypes named
Voyager and Cassini. As well as being optical transponders with integrated packet switching functions,they are characterized by the use of nonproprietary53

Global Standardization Activities

Table 1. Access Projects.
Project Group

Scope and Targets*

Chair

The Edge Computing Project Group will focus on lab and field implementations for services/
applications at the network edge, leveraging open architecture, libraries, software stacks, andmobile edge computing (MEC), into a platform.

Intel
Telefónica

Power and Connectivity

The Power and Connectivity Project Group will initially focus on enabling network operators to
deploy connectivity in areas that do not have electricity. It will also provide an ecosystem in whichconnectivity and electricity providers can collaborate to pilot and scale innovative technology andbusiness models.

Telefónica
Facebook

System Integration andSite Optimization

The System Integration and Site Optimization Project Group will address system integration via
innovative, cost-effective and efficient end-to-end solutions in order to serve both rural and urban
regions in optimal and profitable ways.

Deloitte
Facebook

OpenCellular

The OpenCellular Project Group will focus on wireless access platforms (including cellular) andrelated technologies. It will develop solutions running on the OpenCellular platform, and support
an ecosystem of contributors, OEMs, distributors and system integrators.

Facebook

Solutions Integration

The Solutions Integration Project Group will develop an open radio access network (RAN)architecture by defining open interfaces between internal components and focusing on the lab
activity with various companies for multi-vendor interoperability.

SK Telecom

OpenRANThe OpenRAN Project Group’s main objective is the development of fully programmable RAN
solutions based on general purpose processing platforms (GPPPs) and disaggregated software
so they can benefit from the flexibility and faster pace of innovation capable with software-driven
development.

VodafoneIntel

CrowdCell

The CrowdCell Project Group will focus on creating a CrowdCell by leveraging GPPPs,
software-defined radio and open source designs for both hardware and software to minimize costswith a “one design” flexible platform.

VodafonevRAN Fronthaul

The vRAN Fronthaul Project Group will focus on virtualization of the RAN for non-ideal backhaul,in particular, maximizing the performance through optimization of the physical layer, compression,and other methods.

BT
Vodafone

Edge Computing* Excerpts from the TIP website
OEM: original equipment manufacturer

Table 2. Backhaul Projects.
Project Group

Scope and Targets*

Chair

Millimeter Wave
Network

The Millimeter Wave Network Project Group will define and advance 60-GHz wireless networking
solutions to address the growing demand for bandwidth in dense, highly populated cities by
delivering gigabits of capacity more quickly, easily, and cost-effectively compared to deploying fiber.

FacebookDT

The Open Optical Packet Transport Project Group will define dense wavelength divisionmultiplexing (DWDM) open packet transport architecture that triggers new innovation and avoids
implementation lock-ins. Open DWDM systems include open line system & control, transponder
and network management, and packet-switch and router technologies.

Facebook

Open Optical Packet
Transport* Excerpts from the TIP website

components. In Cassini, for example, a merchantdigital signal processor is employed to implement
coherent optical transmission, which NTT’s laboratory has developed with its partners and is now soldby NTT Electronics. The operating system employed
in it is also a commercial, nonproprietary one.
This type of equipment, referred to as white box,
enables multi-vendor implementation in such a way
as to enable specific parts to be replaced with those ofanother vendor. Although the usage of Voyager orCassini seems to be limited to a point-to-point interNTT Technical Review

Vol. 16 No. 11 Nov. 2018

connection between datacenters, the trend of disaggregation that they embody is likely to penetrate into
more complicated optical transmission equipment.
In the field of wireless transmission, the Millimeter
Wave Network PG is working to establish an infrastructure technology using 60-GHz radio waves. The
frequency falls in an unlicensed band in many countries, including Japan and other Asian countries,
Europe, and North and South American countries,
reducing hurdles for its use. The wireless infrastructure can also be installed more easily than optical54

Global Standardization Activities

Table 3. Core and Management Projects.
Project Group
Artificial Intelligenceand Applied
Machine Learning
End-to-End Network
Slicing
People and Process

Scope and Targets*
The Artificial Intelligence and Applied Machine Learning Project Group will define and share reusable,
proven practices, recipes, models, and technical requirements for applying artificial intelligence andmachine learning to reduce the cost to plan and operate telecommunications networks.
The End-to-End Network Slicing Project Group will identify end-to-end use cases that can beresearched, developed, and demonstrated to help operators overcome many of the key challenges of
employing network slicing to support their 5G services.
The goal of the People and Process Project Group is to share cultural and process transformation
practices that can materially improve operators’ key metrics.

Chair
DT
TelefónicaBT
HPE
Facebook
Bell Canada* Excerpts from the TIP website

fiber. With these advantages, the PG is planning afield trial to obtain practical expertise to apply thetechnology in city areas where capacity demands
tend to increase in a short period.
5. Core and Management Projects
The goal of PGs focusing on the Core and Management field is to innovate network operations. Their
activities are summarized in Table 3 [1].
The End-to-End Network Slicing PG deals with
virtualized networks, or network slices, that are built
on physical infrastructure. Their goal is to achieve
operation that involves configuration and management of network slices spanning multiple operators’domains. To that end, they study interface specifications that enable cooperation of multiple orchestrators, each of which manages network slices and theirconstituent physical resources in a specific domain.
Their scope is aligned with emerging 5G mobile technologies.
The Artificial Intelligence and Applied Machine
Learning PG is working to introduce artificial intelligence (AI) in network operations. They study how
to streamline operation processes or how to optimize
networks by utilizing AI where human judgment currently intervenes. Although their work is in an earlyphase, it could attract attention as a new approach for
introducing innovation.
6. Outlook
The advancement of processor chips’ capabilities
and software engineering techniques and the trend of

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

openness have brought various innovations, disaggregation being an example. Disaggregation, or
unbundling functions or parts that used to be integrated, sometimes as a black box, was first introduced in datacenters.
Because telecom networks contain a wide varietyof equipment in terms of both functionality and size,
it is not likely that the same techniques will be directly introduced into telecom networks immediately.
However, it is almost certain that the approach will
have a significant influence on the research anddevelopment of telecom networks.
NTT has been participating in several PGs of TIP
not just to observe their activities but also to makecontributions. For example, in the Open Optical
Packet Transport PG, we have proposed a packet
switching and forwarding architecture called MultiService Fabric (MSF) [2, 3] that enables highly reliable, multi-vendor implementations. Through theactivities at TIP, we hope to keep engaging ourselvesin the newest technical trends and use them in our
research and development.
References
[1]
[2]

[3]

Excerpts from TIP website, https://telecominfraproject.com/
K. Takahashi, H. Yoshioka, K. Ono, and T. Iwai, “Promoting the MSF
Architecture for Flexible Networks,” NTT Technical Review, Vol. 14,
No. 10, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201610fa6.html
Multi-Service Fabric (MSF), https://github.com/multi-service-fabric/
msf

Trademark notes

All brand names, product names, and company names that appear in thisarticle are trademarks or registered trademarks of their respective owners.

55

Global Standardization Activities

Takeshi Kinoshita

Research Engineer, Media Innovation Laboratory, NTT Network Innovation Laboratories.
He received a B.E. and M.E. in nuclear engineering from Kyoto University in 1994 and 1996.
After joining NTT Optical Network Systems
Laboratories in 1996, he studied optical fiber
communications systems in access networks andtheir management. He then moved to the R&D
center of NTT WEST, where he was responsiblefor developing and introducing commercial services, including broadband Internet access,Internet protocol (IP) v4/v6 virtual private networks, and wide area Ethernet. He also worked
on reducing energy consumption of datacentercomponents such as servers, communications
equipment, and air-conditioning systems. His
current research involves network softwarization
technologies including software-defined networking (SDN) and network virtualization.

Katsuhiro Shimano

Senior Manager, NTT Network Innovation
Laboratories.
He received a B.S. in physics from Waseda
University, Tokyo, in 1991 and an M.S. in physics from the University of Tokyo in 1993. Sincejoining NTT in 1993, he has studied optical networks and network management and relatedareas such as optical network management systems, GMPLS (generalized multiprotocol label
switching), and traffic engineering. He also spenttime at the headquarters working on NGN (NextGeneration Network) network architecture fromthe first phase of construction. He has recently
been leading the research on SDN and network
virtualization at NTT Network Innovation Laboratories.

Hitoshi Masutani

Senior Research Engineer, Media Innovation
Laboratory, NTT Network Innovation Laboratories.
He received a B.E. in communication engineering in 1999 and an M.E. in electrical, electronic, and information engineering from Osaka
University in 2001. After joining NTT Network
Innovation Laboratories in 2001, he studied multicast networking and SIP (session initiation
protocol)-based home networking. In 2005, He
moved to the Visual Communication Division of
NTT Bizlink, where he was responsible fordeveloping and introducing visual communication services, including an IP-based high-quality
large scale video conferencing system and a realtime content delivery system on IPv6 multicast.
He also worked on developing their service ordermanagement system and network management
system for video conference services. He has
recently been developing high performance network function software for telecom carrier networks based on network functions virtualization
at NTT Network Innovation Laboratories.

NTT Technical Review

Vol. 16 No. 11 Nov. 201856

Information

Event Report: NTT Communication Science
Laboratories Open House 2018
Tomoharu Iwata, Tomohiro Amemiya,
Hideharu Nakajima, Yoshinari Shirai,
and Yoshifumi Shiraki
Abstract

NTT Communication Science Laboratories Open House 2018 was held in Keihanna Science City,
Kyoto, on May 31 and June 1, 2018. Over 1600 visitors enjoyed 6 talks and 29 exhibits, which included
our latest research efforts in the fields of information and human sciences.
Keywords: information science, human science, artificial intelligence

1. Overview

2. Keynote speech

NTT Communication Science Laboratories (CSLabs) aims to establish technologies that enable heart
to heart communication between people and people,
and between people and computers. We are thus
working on a fundamental theory that approaches theessence of human beings and information, as well as
on innovative technologies that will transform society.
NTT CS Labs Open House has been held annuallywith the aim of introducing the results of the CS
Labs’ basic research and innovative leading-edge
research to both NTT Group employees and visitorsfrom industries, universities, and research institutions
who are engaged in research, development, business,and education.
This year, the Open House was held at the NTT
Keihanna Building in Kyoto on May 31 and June 1,
and over 1600 visitors attended it over the two days.
We prepared many hands-on exhibits to allow visitors
to intuitively understand our latest research results
and to share a vision of the future where new products
based on the research results are widely used. We alsoorganized an invited talk. This article summarizes theevent’s research talks and exhibits.

The event started with a speech by the Vice President and head of NTT CS Labs, Dr. Takeshi Yamada,entitled “Shift to new dimensions—Further initiatives to deepen Communication Science” (Photo 1).
Dr. Yamada mentioned the remarkable progress

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

Photo 1. Dr. Takeshi Yamada delivering keynote speech.

57

Information

Photo 2. Dr. Yasuhiro Takahashi giving research talk.

Photo 3. Dr. Junji Watanabe giving research talk.

being achieved in deep learning, including recent
developments in artificial intelligence (AI) technologies, and said that CS Labs must not only consist of agroup of masters who use these leading technologies
at a high level but also a group of researchers who are
constantly taking on challenges that will open up new
dimensions. He then introduced recent efforts inhuman and information science research at CS Labs.
He discussed how to cope with risks and uncertainties
accompanying these new initiatives by explaining the
Exploration-Exploitation dilemma, which is wellknown in the field of AI research. The dilemma
involves whether to do deeper research in a field wehave already studied (i.e., Exploitation), or to explore
a field about which we have less experience and do
research widely (i.e., Exploration). He declared thatevery researcher at CS Labs who is responsible forlong-term fundamental research will seek to discoverand explore new, unknown expanses of knowledge
boldly and resolutely.

ers,” by Dr. Yasuhiro Takahashi, Media Information Laboratory
Dr. Takahashi focused on a gate-based quantum computer, which has theoretical evidence ofbeing more powerful than today’s computers.
There are many problems in realizing gatebased quantum computers equipped with a sufficient amount of computational resources, but
he nevertheless introduced two of his recent
results demonstrating that gate-based quantum
computers are still powerful even under the
more realistic condition where computational
resources are limited (Photo 2).
(2)	“Role of haptics in improving wellbeing—Science and design of touch can enhance humanflourishing,” by Dr. Junji Watanabe, Human
Information Science Laboratory
Dr. Watanabe introduced a recently launchedstudy on wellbeing. Wellbeing is gathering
attention because research on designing mindenriching information technology has been pursued recently from a standpoint that is different
from the one aiming to merely achieve greater
efficiency. He introduced several topics including measuring and specifying factors of wellbeing and modifying cognitive and empathetic
attitudes towards oneself and others with haptic
experiences (Photo 3).
(3)	“Beyond combinatorial explosion—Enumeration and optimization with Binary Decision
Diagrams,” by Dr. Masaaki Nishino, Innovative
Communication Laboratory
Dr. Nishino focused on a phenomenon called

3. Research talks
Four research talks were given, as summarized
below, which highlighted recent significant research
results and high-profile research themes. Each presentation introduced some of the latest research
results and provided some background and an overview of the research. All of the talks were very wellreceived.
(1)	“The real worth of quantum computers withelementary operations—Analysis of computational power of gate-based quantum computNTT Technical Review

Vol. 16 No. 11 Nov. 201858

Information

Photo 4. Dr. Masaaki Nishino giving research talk.

Photo 5. Dr. Sadao Hiroya giving research talk.

a combinatorial explosion, in which the numberof possible combinations increases exponentially with the number of elements, which wehave often experienced in a grouping or travel
routes search. He explained how the number of
combinations created by the combinatorial
explosion can be counted by algorithms basedon the data structure called a binary decision
diagram (Photo 4).
(4)	“Speech production and perception share common brain pathways—Investigation of themechanisms of speech communication by
speech conversion and brain imaging,” by Dr.
Sadao Hiroya, Human Information Science
Laboratory
Dr. Hiroya introduced research on neuralmechanisms underlying speech communicationand showed the existence of the common pathways in the brain between speech production
and perception, even though it has been considered that the pathways are different. He also
discussed the difference between human brain
mechanisms and speech recognition and synthesis techniques (Photo 5).

Each exhibit was housed in a booth and employedtechniques such as slides on a large-screen monitor or
hands-on demonstrations, with researchers explaining
the latest results directly to visitors (Photos 6 and 7).
The following list, taken from the Open House website, summarizes the research exhibits in each category.

4. Research exhibits
The Open House featured 29 exhibits displaying
CS Labs’ latest research results. We categorized theminto the following four areas: “Science of Machine
Learning,” “Science of Communication and Computation,” “Science of Media Information,” and “Science of Humans.”
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

4.1 Science of Machine Learning•	Finding similar voice recordings in big data—
Graph index-based audio similarity search•	Learning feature combinations from multiple
tasks—MOFM: low-rank regression for learningcommon factors•	Where do they come from? Where are they
going?—“Data assimilation and navigation
learning for crowd”•	Datafying cities—Event analysis by environmental sensing and machine learning•	Memory efficient deep learning for mobile
devices—Quantized neural networks for modelcompression•	Optics makes machine learning much faster—
Photonic reservoir computing for high-speed
machine learning•	Interpreting deep learning from network structure—Detecting communities in trained layered
neural networks
4.2 Science of Communication and Computation•	Can I borrow your quantum memory?—Highspeed quantum computations with uninitialized
qubits59

Information

Photo 6. The latest research results were exhibited.

Photo 7. Researchers explaining a demonstration.

•	Designing fault-tolerant networks—Maximizing
network reliability via binary decision diagrams•	Can computer translate considering context?—
Context understanding tests for neural machinetranslation•	Early vocabulary development in late talkers—
Collecting and analyzing data from pediatric
NTT Technical Review

Vol. 16 No. 11 Nov. 2018

medical fields•	Chatting with robots broadens your knowledge—Integration of chat and QA based on tworobot coordination•	Anytime, anywhere, we can speak like anative!—Speech rhythm conversion by mobile
application
60

Information•	Sharing enthusiasm between remote sites—
Applause coding for bi-lateral immersive sharing
4.3 Science of Media Information•	Illumination-based color saturation control—
Spectral operation using color enhancement factors•	Pay attention to the speaker you want to listen
to—Computational selective hearing based on
deep learning•	Solving two-choice questions makes AI clever—Deep pairwise comparison model for ASR
hypothesis selection•	Estimating objects’ visuals only from audio—
Cross-media scene analysis•	Cast shadows add dimensions—Projection mapping giving depth illusion to real objects•	Converting impression and intelligibility ofspeech—Speech attribute conversion using deep
generative models•	Creating favorite images with selective decisions—Hierarchical image analysis and synthesis with DTLC-GAN
4.4 Science of Humans•	Predicting attention to the ears by the eyes—
Auditory spatial attention revealed as pupillary
response•	Measuring, understanding, and empowering
wellbeing—Cross-disciplinary research toward“eudaimonic wellbeing”•	Understanding human hearing with AI—Analyzing auditory neural mechanisms with machinelearning•	How do excellent batters hit the ball?—Cognitive processes revealed by body movements inbatting•	How do excellent batters look at the ball?—Cognitive processes revealed by eye movements inbatting•	The sooner you decide, the better you can localize—Visual motion processing in the perception

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

and action•	Let’s FEEL shape and action by a force—Can
we receive environmental information by BuruNavi4?
•	Feeling bumps on a flat sheet—Magnetic haptic
printing technology
5. Invited talkThis year’s event also featured an invited talk by Dr.
Hiroshi Nakagawa, group director, Artificial Intelligence in Society Research Group, RIKEN Center for
Advanced Intelligence Project. The title of his talkwas “AI, ethics and social impact.” He explained ethics guidelines for developing AI systems proposed indomestic and international organizations, personal
data protection in big data utilization, and the proper
way to have a relationship with AI. He also discussed
the hot topic of whether AI will take away all ourjobs. He pointed out that natural language processing
will be one of the most important research fields for
future AI technologies and explained the importanceof redefining what our jobs are by ourselves from ahigher viewpoint.
6. Concluding remarks
Just like last year, many visitors came to NTT CS
Labs Open House 2018 [1, 2] and engaged in lively
discussions on the research talks and exhibits andprovided many valuable opinions on the presented
results. In closing, we would like to offer our sincere
thanks to all of the visitors and participants who
attended this event.
References[1][2]

Website of NTT Communication Science Laboratories Open House2018 (in Japanese).
http://www.kecl.ntt.co.jp/openhouse/2018/index.html
Website of NTT Communication Science Laboratories Open House2018 (in English).
http://www.kecl.ntt.co.jp/openhouse/2018/index_en.html

61

Information

Tomoharu Iwata
Distinguished Researcher/Senior Research
Scientist, Ueda Research Laboratory, NTT Communication Science Laboratories.
He received a B.S. in environmental information from Keio University, Kanagawa, in 2001,
an M.S. in arts and sciences from the Universityof Tokyo in 2003, and a Ph.D. in informatics
from Kyoto University in 2008. In 2003, hejoined NTT Communication Science Laboratories. From 2012 to 2013, he was a visiting
researcher at University of Cambridge, UK. His
research interests include data mining andmachine learning.

Yoshinari Shirai
Senior Research Scientist, Learning and Intelligent Systems Research Group, Innovative
Communication Laboratory, NTT Communication Science Laboratories.
He received a Ph.D. in engineering from the
Graduate School of Engineering at the Universityof Tokyo. His research interests include historyenriched environments, ubiquitous computing,and interaction design.

Tomohiro AmemiyaDistinguished Researcher/Senior Research
Scientist, Sensory and Motor Research Group,
Human Information Science Laboratory, NTT
Communication Science Laboratories.
He received a B.S. and M.S. in mechanoinformatics from the University of Tokyo in 2002
and 2004, and a Ph.D. in biomedical information
science from Osaka University in 2008. He
joined NTT in 2004. He was an honorary
research associate at the Institute of Cognitive
Neuroscience, University College London, in
2014–2015. He researches human-computerinterfaces applying sensory illusions and haptic
perception.

Yoshifumi Shiraki
Research Scientist, Moriya Research Laboratory, NTT Communication Science Laboratories.
He received a Ph.D. in engineering from Tokyo
Institute of Technology in 2015. His research
focuses on signal processing in sensor networks
and visible light communication.

Hideharu Nakajima
Research Scientist, Interaction ResearchGroup, Innovative Communication Laboratory,
NTT Communication Science Laboratories.
He received a Ph.D. in science from Waseda
University, Tokyo, in 2010. His research interests
include prosodic/linguistic/pragmatic analysis of
spoken/written messages, spoken language processing (speech recognition, speech synthesis),speech communication with robots/agents, andeducational technology.

NTT Technical Review

Vol. 16 No. 11 Nov. 201862

Short Reports

Ultrahigh-speed Integrated Circuit
Capable of Wireless Transmission of
100 Gbit/s in the 300-GHz Band
1. Introduction

2. Research backgroundNTT and Tokyo Institute of Technology havejointly developed an ultrahigh-speed integrated circuit (IC) for wireless front-end that operates on aterahertz frequency band, and they have succeeded indeveloping the world’s fastest 100-Gbit/s wirelesstransmission data rate in the 300-GHz band.
Unused terahertz waves* are expected to be applicable to high-speed wireless transmission since awide frequency band can be secured. In our research,
we implemented a mixer circuit that applied a uniqueproprietary high isolation design technology with an
Indium phosphide high electron mobility transistor(InP-HEMT). This enlarged the transmission bandwidth, which is a problem in the conventional
300-GHz-band wireless front end. It also improved
the signal-to-noise ratio (SNR). In addition, we used
this circuit to develop a 300-GHz-band wirelessfront-end module, and we achieved wireless transmission of 100 Gbit/s.
In this research, we achieved 100-Gbit/s wirelesstransmission with one wave (one carrier), so in the
future, we can extend this idea to multiple carriers bymaking use of the wide frequency band of 300 GHz
and using spatial multiplexing technology such as
multiple-input multiple-output (MIMO) and orbital
angular momentum (OAM).
Ultrahigh-speed IC technology is expected toenable high-capacity wireless transmission of 400
Gbit/s. This is about 400 times that of the current LTE(Long-Term Evolution) and Wi-Fi, and 40 times thatof 5G (fifth-generation mobile communications system) technology. Ultrahigh-speed ICs are also
expected to open up utilization of the unused terahertz wave frequency band in the communications
field and in non-communications fields.

High-capacity wireless transmission technology of
100 Gbit/s has attracted worldwide attention with the
spread of broadband networks. There are three waysof further increasing the capacity of wireless transmission—expanding the transmission bandwidth,
increasing the modulation multi-level number, andincreasing the spatial multiplexing number. To realize
future large capacity wireless transmission technology from a level of 400 Gbit/s to that of one terabit per
second (Tbit/s), it is necessary to expand both thetransmission bandwidth and the modulation multilevel number simultaneously in one wave (one carrier) and to increase the number of spatial multiplexing
transmissions by superimposing them multiple times.
The transmission bandwidth is limited in the carrier
frequencies from 28 GHz to 110 GHz that are currently being researched and developed. Thus,
researchers are studying the use of frequencies thatmake it easier to expand the transmission band area,from the 300-GHz band to the terahertz wave frequency band. The 300-GHz band has a frequency thatis 10 times or more higher than the 28-GHz band thatis being studied for 5G, which will be the next generation mobile communications technology. With the
300-GHz band, it will be easier to secure a wide
transmission bandwidth. However, at high frequencies, leakage of unnecessary signals tends to occur
between the ports inside the IC and the mounting, and

NTT Technical Review

Vol. 16 No. 11 Nov. 2018*

Terahertz wave: Just as we use the term kilo to mean 103, so we
use the term giga to mean 109 and tera to mean 1012. Hertz (Hz)is a unit of a physical quantity called frequency. It indicates how
many times alternating electric signals and electromagnetic
waves change polarity (plus and minus) per second. That is, one
terahertz (1 THz = 1000 GHz) is the frequency of the electromagnetic wave in which the polarity changes 1 × 1012 times per
second. In general, a terahertz wave often indicates an electromagnetic wave of 0.3 THz to 3 THz.

63

Short Reports

300-GHz mixer module

300-GHz-band mixer IC

LO (WR3.4)

LO
Bias
G
VG

Bias

IF

RF (WR3.4)
IF (V connector)

RF
Chip size: 1 mm × 1 mm
IF: intermediate frequency port
LO: local oscillation frequency port
RF: radio frequency port

Fig. 1. Mixer IC and module.

Measurement setup
AWG

300 GHz
Tx

DSO

2.22 m
ATT

300-GHz-band Tx

50
dBi

50
dBi

300 GHz
Rx

DSO

na)

ten

na

ten

2

AWG

2.2

ATT

An
m(

an
-to-

Rx Antenna

300-GHz-band Rx
Tx Antenna
ATT: attenuator
AWG: arbitrary waveform generator
DSO: digital storage oscilloscope

Rx: receiver
Tx: transmitter

Fig. 2. Transmission experiment.

so far, it has been impossible to obtain a sufficiently
high SNR. For this reason, even if a 300-GHz band isused, it is impossible to obtain both a wide transmission bandwidth and a high modulation multi-levelvalue at the same time, and so wireless transmission
up to now has remained at the rate of several tens ofgigabits per second.
3. Research results
In this research, we devised our own high isolation
design technology and applied this technology to amixer circuit, which is a key component responsible
for frequency conversion in the 300-GHz-band wireless front end. We also developed an IC with an InPNTT Technical Review

Vol. 16 No. 11 Nov. 2018

HEMT. By applying high isolation design technology, we succeeded in suppressing leakage of unnecessary signals inside each IC and between ports in the
IC. We also succeeded in improving signal noise
reduction and expanding the bandwidth, which had
been issues confronting the use of conventional
300-GHz-band wireless front-end technology until
now. With these technologies, we also developed a
300-GHz-band wireless front-end module (Fig. 1),
and we confirmed reception of a good 16QAM(quadrature amplitude modulation) signal in back-toback transmission. In the 300-GHz band, we also
confirmed transmission at a speed of 100 Gbit/s, thefirst time in the world this speed has been attained(Fig. 2).
64

Short Reports

4. Future prospects
In our research, we achieved 100-Gbit/s transmission with one wave (one carrier), so in the future wewill expand our research to use multiple carriers or
spatial multiplexing technology such as MIMO andOAM together with this result. With this combination, we expect to achieve ultrahigh-speed IC technology that enables high-capacity wireless transmission of over 400 Gbit/s. In addition, we expect thatNTT Technical Review

Vol. 16 No. 11 Nov. 2018

our approach will be applicable in various fields suchas imaging and sensing in which terahertz waves are
expected to be used. Through collaboration with partners, NTT aims to create new services and new industries using ultrahigh-speed ICs, and also aims to further develop ultrahigh-speed IC technology.
For Inquiries
NTT Science and Core Technology Group
http://www.ntt.co.jp/news2018/1806e/180611a.html

65External Awards
Research Encouragement Award
Winner: Hitoshi Niigaki, Ken Tsutsuguchi, and Tetsuya Kinebuchi,
NTT Media Intelligence Laboratories
Date: June 22, 2018
Organization: The Institute of Image Electronics Engineers of Japan
For “Detection of Tree Trunk from 3D Point-cloud Measured by
Mobile Mapping System –Feature Extraction for Sweep Object
Detection–.”Best Paper Award
Winner: Kyoko Sudo, Toho University; Kazuhiko Murasaki, Tetsu­
ya Kinebuchi, Media Intelligence Laboratories; Shigeko Kimura,
Kayo Waki, and Kazuhiko Ohe, The University of Tokyo
Date: July 15, 2018
Organization: Multimedia on Cooking and Eating Activities,
Human Communication Group, the Institute of Electronics, Information and Communication Engineers (IEICE)
For “Intuitively Estimating the Healthiness of Meals from Their
Images.”Published as: K. Sudo, K. Murasaki, T. Kinebuchi, S. Kimura, K.
Waki, and K. Ohe, “Intuitively Estimating the Healthiness of Meals
from Their Images,” Proc. of the Joint Workshop on Multimedia for
Cooking and Eating Activities and Multimedia Assisted Dietary
Management, Stockholm, Sweden, July 2018.
IEEE Photonics Society Distinguished Lecturer
Winner: Hiroki Takesue, NTT Basic Research Laboratories

Date: July 18, 2018
Organization: The Institute of Electrical and Electronics Engineers(IEEE) Photonics Society
For “Coherent Ising Machine: A Photonic Ising Model Solver
Based on Degenerate Optical Parametric Oscillator Network.”
Suzuki Memorial Award
Winner: Shota Orihashi, NTT Media Intelligence Laboratories
Date: August 30, 2018
Organization: The Institute of Image Information and Television
Engineers (ITE)For “A Study on Fast Block Partitioning in HEVC Using Convolutional Neural Network” (in Japanese).
Published as: S. Orihashi, S. Kudo, M. Kitahara, and A. Shimizu, “A
Study on Fast Block Partitioning in HEVC Using Convolutional
Neural Network,” ITE Annual Convention, 21B-3, Tokyo, Japan,
Aug./Sept. 2017.
IPSJ Yamashita SIG Research Award
Winner: Hiroyuki Kirinuki, NTT Software Innovation Center
Date: August 30, 2018
Organization: Information Processing Society of Japan (IPSJ)
For “Automatic Locator Repair on GUI Test Scripts.”
Published as: H. Kirinuki, H. Tanno, and K. Natsukawa, “Automatic
Locator Repair on GUI Test Scripts,” Proc. of IPSJ/SIGSE Software
Engineering Symposium (SES2017), Tokyo, Japan, Aug./Sept. 2017.

Papers Published in Technical Journals andConference ProceedingsInteractive Proofs with Polynomial-time Quantum Prover
for Computing the Order of Solvable Groups
F. Le Gall, T. Morimae, H. Nishimura, and Y. Takeuchi
Proc. of the 43rd International Symposium on Mathematical Foundations of Computer Science (MFCS 2018), Vol. 117, No. 1, pp.
26:1–26:13, Liverpool, UK, August 2018.
In this paper we consider what can be computed by a user interacting with a potentially malicious server, when the server performs
polynomial-time quantum computation but the user can only performpolynomial-time classical (i.e., non-quantum) computation. Understanding the computational power of this model, which corresponds
to polynomial-time quantum computation that can be efficiently
verified classically, is a well-known open problem in quantum com-

NTT Technical Review

Vol. 16 No. 11 Nov. 2018

puting. Our result shows that computing the order of a solvablegroup, which is one of the most general problems for which quantum
computing exhibits an exponential speed-up with respect to classicalcomputing, can be realized in this model.

Novel Optimizing Technique for Linear Optical MachZehnder Modulator and Its Experimental Verification
Using PAM-8 Signal
H. Kawakami, H. Yamazaki, and Y. Miyamoto
Proc. of the 44th European Conference on Optical Communication(ECOC 2018), We2.13, Rome, Italy, September 2018.

66

Papers Published in Technical Journals and Conference Proceedings

We propose an optimizing technique for a linear optical modulator
for selecting whether to linearize the optical power or electric field
response. With this technique, a PAM-8 signal was successfully generated with a driving signal having a swing voltage of 0.7Vpi.

Quantum Computational Universality of HypergraphStates with Pauli-X and Z Basis Measurements
Y. Takeuchi, T. Morimae, and M. Hayashi
arXiv:1809.07552 [quant-ph], September 2018.
Measurement-based quantum computing is one of the most promising quantum computing models. Among various universal resource
states proposed so far, the Union Jack state is the best in the sense thatit requires only Pauli-X, Y, and Z basis measurements. It was open
whether only two Pauli bases are enough for universal measurementbased quantum computing. In this paper, we construct a universal
hypergraph state that only requires X and Z-basis measurements. We
also show that the fidelity between a given state and our hypergraphstate can be estimated in polynomial time using only X and Z-basis
measurements, which is useful for the verification of quantum computing. Furthermore, in order to demonstrate an advantage of ourhypergraph state, we construct a verifiable blind quantum computing
protocol that requires only X and Z-basis measurements for the client.

Verification of Many-qubit States
Y. Takeuchi and T. Morimae
Proc. of the 18th Asian Quantum Information Science Conference(AQIS 2018), p. 48, Nagoya, Aichi, Japan, September 2018.
Verification is a task to check whether a given quantum state isclose to an ideal state or not. In this paper, we show that several manyqubit states can be verified with only single-qubit Pauli measurements. Specifically, we introduce protocols for ground states of
Hamiltonians, quantum states generated by some quantum circuits,
and all polynomial-time-generated hypergraph states. Importantly,
we do not make any assumption that the i.i.d. copies of the samestates are given. Our protocols work even if entanglement is createdamong copies in any artificial way. As an application, we consider theverification of the Bremner-Montanaro-Shepherd-type IQP circuits.

Proc. of AQIS 2018, p. 49, Nagoya, Aichi, Japan, September 2018.
We study the computational power of shallow quantum circuitswith O(log n) initialized and nO(1) uninitialized ancillary qubits,
where n is the input length and the initial state of the uninitialized
ancillary qubits is arbitrary. First, we show that such a circuit cancompute any symmetric function on n bits that is computable by auniform family of polynomial-size classical circuits. Then, we regard
such a circuit as an oracle and show that a polynomial-time classicalalgorithm with the oracle can estimate the elements of any unitarymatrix corresponding to a constant-depth quantum circuit on n
qubits. Since it seems unlikely that these tasks can be done with only
O(log n) initialized ancillary qubits, our results give evidences that
adding uninitialized ancillary qubits increases the computational
power of shallow quantum circuits with only O(log n) initialized
ancillary qubits. Lastly, to understand the limitations of uninitialized
ancillary qubits, we focus on sub-logarithmic-depth quantum circuitswith them and show the impossibility of computing the parity function on n bits.

Resource-efficient Verification of Quantum Computing
Using Serfling’s BoundY. Takeuchi
AQIS 2018 Kyoto satellite workshop on quantum computing,
Kyoto, Japan, September 2018.
In measurement-based quantum computing, checking whether
correct graph states are generated or not is essential for reliable quantum computing. Several verification protocols for graph states havebeen proposed, but none of these are particularly resource efficient:
Many copies are required in order to extract a single state that is
guaranteed to be close to the ideal graph state. For example, the best
protocol currently known requires O(n15) copies of the state, where nis the size of the graph state. In this talk, we propose a significantly
more resource-efficient verification protocol for graph states thatneeds only O(n5 log n) copies. The key idea that achieves such a
drastic improvement is to employ Serfling’s bound, which is a probability inequality in classical statistics. Utilizing Serfling’s boundalso enables us to generalize our protocol for qudit graph states andcontinuous-variable weighted hypergraph states. This talk is based on
joint work with Atul Mantri, Tomoyuki Morimae, Akihiro Mizutani,
and Joseph F. Fitzsimons. The detail is given in arXiv:1806.09138.

Power of Uninitialized Qubits in Shallow Quantum Circuits
Y. Takahashi and S. Tani

NTT Technical Review

Vol. 16 No. 11 Nov. 201867