NTT Technical Review

March 2020 Vol. 18 No. 3

View from the TopAtsuko Oka, Senior Vice President, Head of Technology Planning, NTT

Front-line ResearchersTessei Kobayashi, Senior Distinguished Researcher, NTT Communication Science Laboratories

Feature Articles: NTT R&D Forum 2019 Special Sessions
A Digital World of Humans and Society—Digital Twin Computing

Innovative Network for 2030 (Beyond 2020)

Basic Technologies toward the All-Photonics NetworkUpgrade Reality—Reality in IOWN Concept

Feature Articles: ICT R&D Initiatives for a Smart World that Harmonizes People andEnvironments

R&D Initiatives to Make the Most of Human Abilities by Amplifying Human Intelligence toAchieve a Smart World that Harmonizes People and EnvironmentsNatural Communication Technologies Providing Information through Natural Exchanges thatDo Not Interfere with Human Activities



Ambient Assistance Technology Supporting Human Behavior by Understanding theRelationship between People and Environments

User Experience Method for Deeper Understanding People’s Values to Design and Change
Their Behavior

Sports Social View—Method of Inclusive Sports Appreciation Extracting and Sharing the
Essence of Sports through the Sense of Touch

Regular Articles


Field Trials of Open and Disaggregated Transponder in Optical and IP Networking

Does Exploratory Hand Movement Provide Information on Tactile Sensations?

Global Standardization Activities


Activities at W3C Technical Plenary and Advisory Committee Meetings Week (TPAC) 2019 inFukuoka

Short Reports

NTT and Telefónica Start PoC on Open Optical and Packet Transport Technologies within
Telecom Infra Project

External Awards/Papers Published in Technical Journals and Conference Proceedings

View from the Top

Promoting Communication with
Digital Transformation and Applying
Mutually Exclusive and Collectively
Exhaustive Principle
Atsuko Oka
Senior Vice President, Head of Technology
Planning, NTT
Overview

The NTT Group is committed to solving social issues, and its initiatives
and responsiveness have been highly evaluated. The Group joined the international environmental protection initiatives EP100 and EV100 as their firsttelecommunications carrier member. It has also been selected as a global
top innovator for the eighth consecutive year. We asked Atsuko Oka, NTTsenior vice president, head of Technology Planning, for her 2020 outlook
for the NTT Group, which strives to provide high-quality, highly reliable
services for safe and secure operation of information and communication
technology infrastructure.
Keywords: digital transformation, IOWN, MECE

Providing reliable communication infrastructure
as Your Value Partner—What kind of year was 2019?
In 2019, typhoons Faxai and Hagibis hit theJapanese archipelago in quick succession and caused
severe damage in many areas due to strong winds andriver flooding. I’d like to express my deepest sympathies to all those affected.
About 80,000 communication lines were affected.
While NTT was working to restore telecommunications services, which is a social infrastructure, I
strongly felt that we need to consider strengthening
our resilience (namely, ability to recover from disasters and failures)—starting with quick recovery—and
further strengthen dissemination of information. We
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

have thus begun efforts to (i) predict the impact oncommunication facilities from the path and intensityof a typhoon and (ii) prepare in advance people andmaterials necessary for restoration near areas where
damage is expected. Since those efforts are based on
prediction, they may end up in vain. Even so, from thestandpoint of a telecommunications carrier, quick
recovery is our top priority, and I think it is too late tostart responding after a disaster. To make it possible
to quickly restore the infrastructure, we are making
full use of artificial intelligence and other technologies developed by NTT laboratories to predict theoccurrence of disasters and respond proactively tothem.
I thought it was necessary to disseminate not justinformation about disasters affecting our communication facilities but also information about what kind1View from the Topof services NTT provides, what kind of equipment isused for those services, what state the equipment isin, and what recovery activities are ongoing. Major
international events are increasingly being held inJapan, so many visitors come to participate in andwatch these events. Web services and social networking services are not only supported in Japanese but
also in English and many other languages. Since such
events are becoming ever more regular, it is important
to ensure stable communication and tight security sothat they run smoothly and companies operate normally. The Security Strategy Section of the NTT
Technology Planning Department is carrying out
early detection, analysis, and response and investigating technology for early defense regarding cyberthreats. As the saying goes, “Being prepared means
peace of mind;” thus, for stable provision of communication, we want to proactively respond to anythingby paying attention to details.

and communication with people they haven’t spoken
to before will increase further. As a result, open innovation will progress.

—It is encouraging just to hear that. How will you
tackle this challenge in 2020?

—It seems that a new world can be opened up bychanging the way we use time.

NTT announced a medium-term management strategy called Your Value Partner in November 2018.
The pillars of this strategy are twofold: promoting ourown digital transformation (DX) and supporting ourcustomers’ DX. This strategy is a major mission for
us. Although DX sounds like pursuing cost reductionand efficiency, by freeing up more time for people
through DX, it will help to reform work styles such as
reducing overtime work, enhance communication,and promote open innovation. For example, in thecase of a business that conventionally uses paper,
digitizing the entire business process and distributing
digitized information will accelerate the entire process. This digitization will speed up the provision ofservices to customers, thereby improving customer
satisfaction and customer experience. Moreover, we
are aiming to increase profits by providing customers—as a future solution service—with actual examples of us promoting DX and making things moreconvenient.
We are already conducting activities in each field.
We have launched a cloud-based billing platform
called Smart BillingTM and are proposing a cloudbased contract-management platform called Smart
Fulfillment to corporate customers.
With customers seeming to be actively working on
DX, we are often asked to tell them about what we arepromoting at NTT. As customer DX proceeds, companies and individuals will change their work styles,

Looking to the future, we are promoting theInnovative Optical and Wireless Network (IOWN), a
next-generation communication infrastructure that isbeing targeted for practical use around 2030. The
IOWN Global Forum, a new industry forum forachieving IOWN, has already been registered as acorporation, and NTT, Sony, and Intel have been
appointed as initial board members. More than 100
companies in Japan and around the world, including
Microsoft, Verizon, Orange, and China Telecom,
have shown interest in joining. The response to setting up the forum was faster from outside Japan. Until
recently, the concepts that NTT has launched haveoften been disseminated only within Japan. However,the positive reaction from global companies to IOWN
can be attributed to the establishment of new global
bases including NTT Research, Inc. in the UnitedStates, strong interest in the All-Photonics Network,
the key element of IOWN that introduces opticaltechnology end-to-end from the network to the terminal, and the expectation of IOWN as a solution toissues such as limitations of semiconductor devices
concerning low-power consumption and high-speedtransmission.
The NTT Technology Planning Department willcontinue introducing, operating, and maintaining newtechnologies developed in our research laboratories
as well as outside ones while formulating and implementing measures related to disaster countermeasures,

NTT Technical Review

Vol. 18 No. 3 Mar. 20202

View from the Top

security, and efficient procurement of equipment.
When the technologies, new services, and businesses
that make up IOWN become concrete, we will also
consider the grand design of a network suitable for
IOWN as well as a migration plan for gradually
migrating from existing networks to the All-Photonics
Network.
“Practice makes perfect”—Please tell us what you value in carrying out yourmission.
“Practice makes perfect” is my favorite phrase. I
believe that we will be able to handle almost anything
if we do things seriously and steadily. Also, mymaxim is be timely; I am often the first to arrive atmeetings. Other attendees arrange their schedules tofit meetings into their busy schedules; in other words,
they reserve their precious time for meetings. Many,
however, can barely fit meetings into their schedules.
Time is lost when time is not kept. Today, time can bebought with money, such as when we use electricalappliances to cut down on time consumed by household chores; however, I don’t think face-to-face communication and getting involved with challenging
tasks can be bought. I feel that precious time should
not be wasted.
I take notes at work and keep a personal diary. I
don’t write much and it’s not well organized, butsometimes I notice things when I look back. I don’t
think we can rely on memory alone. The beginning of

this habit dates back to my college days. Attending a
combined junior and high school, the people aroundme were all old friends, but when I entered university,
the people around me were all new. At first, in anenvironment where I had nobody to rely on, I realized
I could rely on only myself and started taking copious
notes for my classes. When I lent my notes to classmates before tests, those notes kept circulating
among the students; thus, I contributed in some way
to the student community. This continued into myprofessional life. After joining NTT, I went to work at
an affiliated organization, and I kept taking notes,
even though my team didn’t have the habit of doing
so, and they eventually used my notes. Therefore, it
can be said that the notes and memos I took formyself contributed to my work.
I also maintain a conscious attitude based on the
MECE (mutually exclusive, collectively exhaustive)
principle. If you focus too much on your task, you
won’t see the whole picture, and you will be worried
that you are missing something. However, if you takea step back and look at the whole picture again froma slightly higher viewpoint, you may notice an omission, so I work while being aware of MECE. It is
important to categorize each task by creating a framework to understand the whole picture. This will help
reduce unnecessary meetings and redundancy in mywork. I only got into this habit after starting my professional life. I may have picked it up when I wasworking at the Corporate Planning Department of
NTT Communications. I felt guilty if I had to ask forsomething again because of my narrow or lack ofvision.
—You can learn from anything and apply what you
learned to future tasks! Please tell us about any turning points you encountered.
When I joined NTT, I was assigned to Software
Research Laboratories, where I was engaged inresearch related to the Internet. The sudden transfer tothe International Business Department changed mymindset, and it seems to me now that it was a turning
point.
In the 1990s, the Prime Minister of Malaysia,
Mahathir bin Mohamad, gave his vision for a fullfledged information society arriving in the 21st century, which he called the multimedia super corridor(MSC), which would be achieved while competing
against developed countries around the world. Afterthe MSC project was launched, NTT received arequest for cooperation, and I was transferred to the

NTT Technical Review

Vol. 18 No. 3 Mar. 20203

View from the Top

International Business Department to participate inthat project.
At that time, I was very worried because I could not
speak adequate English. Although I had some experience in presenting my research at conferences, I had
no experience in promoting NTT technologies outside Japan or working with people from different
countries to create concepts, so I asked myself “whatshould I do?”When I started working on the MSC project, however, I realized that NTT’s technologies were outstanding and was driven by the desire to promote
those technologies, so I felt I was moving forward. I
also thought that if I did not present such technologies
effectively, other people would not understand them,and communication was the key to that understanding. On top of that, I realized that it is necessary tohave a marketing mindset like that taught in businessschools. Unexpectedly, this challenging time turned
into an opportunity.
I faced another turning point in 1999. I studied
abroad and learned business basics and skills todevelop business from technology. Through my experiences studying abroad and in Malaysia and working
with multinational team members on projects, I cultivated diverse perspectives and communication skills
and recognized the importance of the spirit of contribution.
“A feeling of being worthwhile” gained byworking hard
—What does “work” mean to you?
To me, work naturally means working hard and getting paid, but at the same time, it means changing theworld through the results of our work and satisfying
our customers with the services we provide. As I said,
my work hasn’t always gone smoothly. For example,
moving to the International Business Department andparticipating in the Malaysian project may have been
a setback in some ways, including a language barrier.
However, there is always a way; we are not working
alone. One of the great things about joining NTT isthat someone always helped me, and I am very grateful for that help.
I tried to find a role model. However, it is not easy
to find people with similar backgrounds or in similar
situations. Therefore, I’ve been trying to find a good

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

role model by “cherry picking” when interacting withvarious people. Sometimes I wonder if my activities
will be helpful to younger women. In fact, the numberof women in managerial positions is increasing, so I
want to observe their work style and performance and“pick” the best of them and take on further challenges.
—Please say a word to researchers and engineersinside and outside the company.
I originally got involved in technology as a
researcher. Technology does not always take shape
immediately and lead to business. Sometimes it takes
shape after some time, maybe even after 20 years.
The first thing I did after returning from studying
abroad was help launch a video-distribution service.
At that time, said to be the first year of broadband,
most video distribution was 64-kbit/s quality.
However, various technologies have been combined,
and now we have reached 5G (fifth-generation
mobile communications), which allows us to enjoy
stunning images on smartphones. Some technological developments produce results immediately, whileothers blossom with time. Accordingly, it is important to make the research and technology that we areworking on open through communication with therest of the world. Unlike in the past, information can
now be transmitted in many ways, such as via theweb, and the importance of open innovation and coworking is increasing. That’s why integrity is fundamental when it comes to making things open as wellas communicating and connecting with people.

4

View from the Top

Interviewee profile
Career highlights
Atsuko Oka joined Nippon Telegraph andTelephone Public Corporation (now NTT) in
1988. After serving as the director of the IP
Services Department of the Net Business Divisionof NTT Communications from 2006 to 2010, shebecame the president and CEO of NTT NaviSpacein 2010 and served as a board member and general manager of the Solution Business Division of
NTT Resonant from 2017 to 2019. She assumed
her current position in June 2019.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

5

Front-line Researchers

In Life, Nothing Is Wasted. Let’s
Build a Society in which Everyone
Can Live an Active Life
Tessei Kobayashi
Senior Distinguished Researcher, NTT
Communication Science Laboratories
Overview
A report published in August 2015 by the World Literacy Foundation estimates that economic and social loss due to illiteracy inJapan is about 95.1 billion yen annually [1]. Thanks to compulsory education, Japan is said to have achieved a literacy rate of
99.8%. However, according to a survey by the National Instituteof Informatics conducted in 2016 [2], many junior- and seniorhigh-school students do not understand the structure of basic
sentences, such as the relationship between subjects and predicates. We interviewed Tessei Kobayashi, a senior distinguishedresearcher at NTT Communication Science Laboratories, who isresearching the development of the linguistic ability, which plays an important role in social participation, about his current research and attitude as a researcher.
Keywords: child language development, early education, picture bookToward educational support based on evidence—Would you like to start by telling us about your current research?
I’m researching the mechanism of child language
acquisition and evidence-based educational support.
Children generally learn the basics of their mothertongue during their first 3 years of childhood. I’m
researching the mechanisms of speech perception,utterance, and learning of vocabulary, grammar, andletters involved in that learning process. English as asecond language has recently been included in elementary schools in Japan, and this area has also been
my research target.
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

I’m also studying social cognition. Children areable to judge the feelings and emotions of others fromaround 3−5 years old, and I’m investigating thedevelopment of that ability in relation to language
skills. The focus has been on both language acquisition and cognitive skills of typically developing children; however, I think that if the feeling/emotionjudgement mechanism is understood, it will be possible to understand the stumbling points concerning
language acquisition in atypically developing children. My goal is to continue researching these topics
in collaboration with medical institutions such as
hospitals and provide evidence-based educational
support.

6

Front-line Researchers• Based on a vocabulary-checklist survey of 1500 parents, a child-vocabulary-development(CVD) database about when children learn is created.

Acquired age data on early Japanese vocabulary

Data collection“Bow-wow”
1.0

Understanding
Utterance• Acquire vocabulary-checklist
data from 1500 0−4 year olds
• Target words: 2688 words

Acquisition rate

0.8

“Dog”
1.0

Understanding
Utterance

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0
0

5

The world’s-largest
database

10 15 20 25 30 35
Age (in months)

15.5 months

05

10 15 20 25 30 35
Age (in months)

26.1 months

Fig. 1. Child-vocabulary-development database.

—It’s a combination of your wisdom and affection forchildren. Your research might help many parents inraising their children. So, specifically, what kind ofresearch do you conduct?
In one of my studies, namely, the mechanism bywhich Japanese vocabulary is acquired, not muchbasic data are available. Accordingly, we asked mothers of children aged up to 4 years to check what words
their children can say at that stage. We then createdstatistical models to understand the initial development of language acquisition. Looking at the relationship between age and language acquisition from thecollected data revealed that about 50% of infants at
15.5 months can say the word “wanwan” (Japanese“bow-wow”) when they see a dog, whereas they cansay “inu” (“dog”) at 26.1 months. Moreover, the firstword that infants can understand is their own name,and about 50% of infants can understand their nameat about 7 months. We confirmed that in the next stepof language acquisition infants understand words
used for social communication such as “peekaboo”
and “bye-bye” and by their first birthday, they canspeak by understanding about 15 words on average.
We thought that it would be useful if we could create
a database of such data and use it to search for words
according to age (in months and years) of infants andcreate content according to their age (Fig. 1). BasedNTT Technical Review

Vol. 18 No. 3 Mar. 2020

on such scientific findings, I’ve been supervising an
educational television program for over 10 years.
Evidence-based guidelines can now be added to theprogram production, which was conventionally basedon years of experience and know-how.
We have also created picture books based on thisdatabase. A total of about 300,000 copies have been
published in a series of five books. Although we created a picture book based only on words learnt at theearlier stages of language acquisition, the rate of language acquisition differs from child to child. Given
that fact, we have been conducting field experimentswith the aim of creating picture books personalized
for individual children. That is, mothers are asked tocheck the vocabulary status of their children, and apicture book incorporating words learnt is createdthat is suitable for that child. Since this picture bookwas very popular among test participants, we started
commercializing it in collaboration with NTT Printing in December 2019 and began test sales as four
personalized educational picture books. Picture
books significantly contribute to language acquisition, so this picture book represents an effort to support language acquisition. In general, as part of programs such as Bookstart, local governments distribute one picture book to parents when infant medical
checkups start; however, it is difficult to get children
into the habit of reading picture books by distributing
7

Front-line Researchers*Sota is a legal trademark registered by Vstone Co., Ltd.

Fig. 2. A child using Pitarie.

Fig. 3. D emonstration of Pitarie-Touch at the Picture Book
Museum.

only one book.
To encourage parents and children to learn languages by making it a habit to read picture books, I
thought it was important to motivate them to go to thelibrary. In January 2020, in cooperation with the village of Onna in Okinawa, Japan, we began creating apersonalized educational picture book for parents andtheir children who received a medical checkup to givechildren opportunities to go to the library. The personalized educational picture book is completed inabout 10 days, so it is a mechanism for getting children to go back to the library to receive their books in
an attempt to make it a habit to read picture books
through repeated behavior, that is, if children areencouraged to return to the library many times, they
can choose suitable picture books from the collection.
We have also developed a system of searching for
picture books—called Pitarie—that searches for picture books that match the interest and the developmental stage of individual children [3] (Fig. 2).
Although a target age group is indicated on each picture book, the range is quite wide. Therefore, by (i)analyzing the target age presented on the picture bookand the word and sentence data appearing in the picture book and (ii) estimating the complexity of thesentences by using academic data and natural language-processing technology, picture books can be
retrieved based on a narrower range of age and readability. As well as the title, the words used in the
picture book are sorted; however, in picture books,
fonts are not in a fixed form and characters may overlap the pictures; thus, picture books cannot be readmechanically using optical character recognition, etc.

Accordingly, 6000 picture books were manually
input to this system. Note that there can be an error
between the target age and actual vocabulary status of
the child depending on the status of the child’s development. Therefore, the target age represents a guideline only. Nevertheless, I think that using Pitarie
makes it possible to select suitable picture books for
an individual child, and get children into the habit ofgoing to libraries and reading picture books.
Pitarie has been commercialized by NTT DATA
Kyushu, and in April 2019, it was officially introduced at the Fukui Prefectural Library. It was thenofficially installed at a nursery school in Shinagawa
ward, Tokyo, and was very popular among the childcare staff. At an event called “Picture Book Museum”
held during the summer break in Fukuoka City, ademonstration of recommending favorite picture
books while interacting with a robot was held, and it
was experienced by 20,000 people over almost two
years (2018−2019) (Fig. 3). It was an experience thatdiffered slightly from everyday life, so it was fun forthe children. During the experience, the process ofdeveloping preferences for colors and patterns wasanalyzed. For example, in the case of a picture of aperson on a book’s cover, it turned out that children
like faces with big cartoon eyes, which differs from
more artistic faces on books that adults want to read.
In the future, I’m thinking of analyzing childgrowth records. With the cooperation of a nurseryschool and growth records regarding the vocabularythat children learned will be input by the staff into a
special app, which could then recommend picture
books suitable for a particular stage of a child’s

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

8

Front-line Researchers

growth. Although we already have the know-how, we
are investigating ways to implement this without putting a burden on childcare staff. More recent studies
in developmental psychology have shown that as thefrequency that parents read picture books aloud totheir children at the age of one or two increases, thereading comprehension and mathematical skills of
third and fourth graders in elementary school (8–9year olds) improves. With the hope that our research
will help support this learning base, we have presenting our results throughout Japan.
Ultimately, enthusiasm and motivation pave theway forwardwith various people. I’ve recently become very excited to handle the entirety of this process even though
it is a small part of the NTT Group. I think this is notpossible at research institutions such as universities.
Originally, I was a scientist specializing in psychology and biology, so I was conducting research without being fully conscious of my contribution to society. However, because I’m working in a company, I
think it is very rewarding to transition researchaccomplishments into businesses in collaboration
with many colleagues and partners such as with Pitarie.
—What is the motivation that supports your research
activities?

—How did this research get started?
I think science is about explaining natural phenomena, and engineering is about creating things that we
can use by referring to the explained phenomena.
Humans speak in words, but animals do not. I wanted
to find out why we can start to speak naturally after
birth; that is, science was the trigger for my research.
At university, I majored in psychology and became
interested in human beings. In graduate school, I
expanded my research to include the psychology ofchimpanzees, elephants, and rats and investigated themechanism of thinking and cognition in humans
before they acquire language (a unique human characteristic) by comparing it with that in mammals.
From that experience, I learned that humans are special among mammals because of traits such as ourlanguage ability, and I have come to explore our existence from an evolutionary and developmental perspective. Approaches to researching language vary
widely. Even so, research on infants enables us to follow the process from knowing nothing up to learninga language; therefore, I thought it was important to beable to see the transitions of development and evolution.
Many scientists find it interesting to take up thechallenge of solving what has not been solved. I havethe same feeling. When I faced an unsolvable problem, could not explain the analytical data as hypothesized, or could not understand the clues, I stillthought that it was worthwhile and pushed on withmy research.
In general, basic research in science attempts to
explain natural phenomena and presents results inpapers. However, my current work has been expanded to the point at which such research results can be
put to practical use after engaging in much discussion
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

My motivation is twofold. The first is the feeling I
get from the moment that I can understand the smallthings that I didn’t understand before, that is, the joyof understanding. The second is the joy of having
someone apply my research results and realizing I
made a contribution. It is up to researchers to decide
what approach to take when they begin research, andI think it depends on their sense of determining which
problem to solve. In my case, I took the problem of
human language acquisition, and how to solve thatproblem also depends on the sense of the researcher.
I want to—I need to—brush up this skill, so it isimportant to take on such challenges.
However, there are times when we are asked questions such as “Does this mean anything?” Throughout
the various stages of research and commercialization,
people in various positions express differing opinions
and concerns. In my experience, many of these opinions and concerns have been valid, which sometimes
stopped me from doing what I tried to do. As you
work on new research topics, getting people to understand them requires a great deal of effort. However, in
any situation, it is important to push ahead withenthusiasm and confidence in what one really thinks
is right. Rather than expecting someone to do something for you, it is necessary to take responsibility andhave the willingness to do it all yourself until the very
end, and this mindset is a source of motivation.
Nothing is useless in life. Get involved with thosearound you and push ahead—Please say a few words to our researchers.
After joining NTT, I was ordered to research a subject different from the specialized field that I was
9Front-line Researchers

involved in as a student. Being able to stick to yourspecialty is very fortunate, but when you are not able
to do that, which is often the case, turn what you are
currently working on into something you like.
Although my current research is completely different
from that when I joined the company, I feel that my
new specialty is interesting. Also, by stepping into
engineering and business areas, I can realize that myresearch results are useful in society.
When I wasn’t able to do what I wanted, I struggled
through every day. I remember the head of the laboratory at that time saying, “There is nothing wasted inyour life.” At that time, I thought it was just words of
comfort, but now I realize it is true. Something seems
to be useless at first glance, but by working on it andgaining experience, it can be very useful regarding
my current research.
As I mentioned earlier, it is important to have those
who support you and get people around you to understand the significance of your research. Research anddevelopment is about creation and revealing new
ideas; therefore, it can be difficult for others to understand. However, by patiently repeating explanations,
you can gradually increase the number of people who
understand your work and support you. By working
with such people, you can produce concrete results.

We still do not fully understand the learning process of writing and reading Japanese hiragana. Since
the only large-scale study on this was conducted 50
years ago, we want to collect the latest data. In a previous study, we got children to write hiragana on
paper, video-recorded it, and analyzed it; however,
we can now measure writing order and speed by having children write hiragana on a tablet device that can
automatically record and analyze such data. Analyzing typically developing children is more likely tolead to support for children with dyslexia. Studying
child development revealed that a small stumble inthe middle stages of development may reverberate
later in life. I believe that if we find this stumble earlyon and provide support and training to address it,even if doing so does not lead to a fundamental solution, the child in question will be able to lead a fulfilling social life. I hope to continue supporting children
so that they can have such a life.

—How will you proceed in the future?

[3]

I think it is my research style to set specific goals,
go one step at a time, and achieve those goals. My
team is currently conducting research on emotional
development. I believe that knowledge of emotions,that is, thinking about and reading what others are
feeling, is a very important ability in regard to children’s social life, so we are measuring this ability andinvestigating how to use it to support emotional
development.
In the near future, we plan to conduct a survey atseveral elementary schools. For example, if emotion
is added to the word “Hello?”, the expression willvary in meaning. We will measure how well children
understand such words. Mainly outside Japan, information on children having difficulty understanding
speech is being gathered. With that in mind, I want tounderstand the actual situation in Japan and consider
how to improve the capacity to understand speech.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

References
[1]

[2]

World Literacy Foundation, “The Economic & Social Cost of Illiteracy—A snapshot of illiteracy in a global context,” https://www.literacyhow.org/wp-content/uploads/2016/02/WLF-FINAL-ECONOMICREPORT.pdf
Press release issued by the National Institute of Informatics on July26, 2016 (in Japanese), https://www.nii.ac.jp/news/release/2016/0726.
html
T. Hattori, T. Kobayashi, S. Fujita, Y. Okumura, and K. Aoyama,“Pitarie: Picture Book Search with Interdisciplinary Approach,” NTT
Technical Review, Vol. 14, No. 7, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201607ra1.html

Interviewee profile
Tessei Kobayashi
Senior Distinguished Researcher, Interaction
Research Group, Innovative Communication
Laboratory, NTT Communication Science
Laboratories.
He received a Ph.D. in psychology from the
University of Tokyo in 2004. He is currently
researching child language development, specifically vocabulary spurts and syntactic bootstrapping.

10

Feature Articles: NTT R&D Forum 2019 Special
Sessions

A Digital World of Humans andSociety—Digital Twin Computing
Ryutaro Kawamura
Senior Vice President, Head of NTT
Service Innovation Laboratory Group
Abstract

This article introduces a lecture presented by Ryutaro Kawamura, senior
vice president and head of NTT Service Innovation Laboratory Group, at a
special session of NTT R&D Forum 2019 held on November 14 and 15,
2019. The lecture described the latest research activities at the laboratorygroup, focusing on NTT’s vision of Innovative Optical and Wireless Network or IOWN.
Keywords: I OWN, Digital Twin Computing, digital representation ofhumans

1. How will humans use the power of digitaltechnology in the future?
Looking forward 10 and 20 years into the future, towhat end will we, humanity, use the power of everadvancing digital technology? As you know, predictions are being made about the coming of a major
turning point in social systems, and Japan’s Society
5.0 initiative reflects such ideas. Discussions are alsoraising the possibility that the long-lived capitalistic
society is coming to an end. Let’s look back at thehistory of digitalization in relation to humans andthings over the past 30–40 years. To begin with, email
appeared around 1985. This development can be seen
as a step forward in digitalization centered aroundhumans. Then, around 1995, the Internet appeared,
and at the same time, the digitalization of information
accelerated in relation to things, which improved
daily life and services such as for providing products,
timetables, and maps. Next, a new era in human communication appeared around 2005 in the form ofsocial networking services (SNSs), and since 2015,
we have been in an era marked by the digitalization ofthings driven by a combination of Internet of Things
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

(IoT) and artificial intelligence (AI). Looking back at
this recent history of digitalization, the digitalizationof humans and things has been progressing in a mutually repeating cycle. Taking this cycle into accountand viewing the recent development of IoT, we consider that it is time for human digitalization to takeanother turn. Of importance here is that value in thisnew era will increase not in a linear and proportionalmanner but rather in an explosive and discontinuous
manner, and I predict that the time for this to occur isimminent (Fig. 1). In this regard, the expression digital twin has recently come into use in the sense thatthe digital representation of humans and things canenable replication, fusion, and exchange as well assaving and recording, all of which are strong points ofdigitalization. However, in much the same way that
so-called silos have appeared in industry, the fact isthat the digital twin concept has progressed in separate industries with no mutual compatibility.
2. Digital Twin Computing initiative
The digital twin framework up to now has been to
map individual targets in the real world represented
11

Feature Articles

Value of digitalization

1995~
Things:1985~
1st generation
Humans:(Internet)
1st generation(communication)
Products,
stores,Emailtimetables,
maps…

2005~
Humans:
2nd generation
(SNS)

2015~
Things:
2nd generation(IoT+AI)• New turn for human digitalization• Value explosion throughcombinations

Cars, factories,
distribution, crops…

Facebook
Twitter
Search logs
Purchase history…

Time

Fig. 1. Progress in digitalization in relation to humans and things.

by automobiles, robots, etc. in cyberspace and to analyze those targets and make predictions. The resultsof such analysis and predictions are then remapped to
the real world and put to use.
As an extension of this conventional digital twinframework, we proposed Digital Twin Computing
and began an initiative to make it a reality (Fig. 2) [1].
Digital Twin Computing takes digital twins of thingsand humans in diverse industries and performs computations on them in any desired combination. This
makes it possible to accurately reproduce combinations that could not be comprehensively handled up tonow, such as humans and automobiles in a city, and to
make predictions about the future. In addition, Digital Twin Computing represents a new computing
paradigm that goes beyond physical reproduction of
the real world by achieving interactive effects among
digital twins including the inner state (e.g., consciousness and thought) of humans in cyberspace.
This initiative will endeavor to configure a virtualsociety composed of a variety of digital twins, replicate in cyberspace digital twins of single entities in
the real world, and exchange or fuse some of the elements constituting different digital twins to generate
new digital twins that do not exist in the real world.
This initiative will take up the challenge of achieving a digital representation of the inner state of individuals. By representing not just the outward appearance of humans but their inner state as well, it should
be possible to achieve advanced interactivity even
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

from a social perspective such as human mobility andcommunication. Moreover, representing the personality of every person should make it possible toachieve interactivity based on diversity and individual features as opposed to interactivity between individual digital twins without personality that are statistically rounded out as average values.
We argue that these features will enable the creationof a virtual society in which a variety of things andhumans interact with each other in advanced andsophisticated ways beyond the limitations of the realworld.
3. Digital representation of humans
It is important to note that a human digital twin in
Digital Twin Computing can provide not only a digital representation of anatomical and physiologicalfeatures but also a digital representation of a person’s
inner state. There are two main approaches to achieving such a difficult objective. The first approach is toemulate human abilities using computers and to
repeat that process to “get continuously closer tohuman qualities.” Technologies for recognizing
sounds and voices and for communicating via conversation are good examples of how progress can bemade with this approach. The second approach,which might be called the ultimate approach, is tophysiologically clarify the human brain and body andtranscribe the results to a computer. This field, which
12Feature Articles

Conventional digital twin framework

Individual use of digital twins

Digital Twin Computing
Autonomous
social systems

Future society
designAutomatic
decisionmaking

Extension ofhuman abilities

Traffic jam

Autonomous
driving

Robot control

Medical careTraffic environment

Urban space

Replication, fusion, exchange

Digital Twin Computing

Digitaltwins

Digitaltwins

ThingsHumansThingsHumansFig. 2. Digital Twin Computing initiative.

is representative of brain and neuroscience, has been
making much progress and has already been producing research results that can be used for engineering
purposes. Our plan is to work toward this digital representation of humans by using the best elements ofthese two approaches (Fig. 3).
The following introduces several key technologies
that NTT laboratories have so far taken up withrespect to the first approach (Fig. 4).
3.1 Speech recognition
NTT laboratories have been researching for half acentury how to accurately recognize the human voice
as a technology for listening [2]. This research beganwith the recognition of words and clearly spoken
sentences, but from around 2010, the technology wasable to accurately recognize natural human utterancesand be used at customer-contact centers. With theintroduction of the latest neural networks, the technology is finally approaching the abilities of humanspeech recognition.
3.2 Speech synthesis
The question here is how and to what extent textual
information can be converted to natural, humanlike
speech. This technology includes text analysis proNTT Technical Review

Vol. 18 No. 3 Mar. 2020

cessing to determine the reading of kanji (Chinesecharacters) in the Japanese language according tocontext and processing for synthesizing speech signals with appropriate pitch and speed. From around1990, the technology has been used for making automatic replies to calls as well as synthesizing speechof animated characters, robots, etc. Deep learning
based on speaker voice data is driving the synthesis ofnatural and diverse voices that have the feel of a real
voice.
3.3 Understanding emotions and intent
Various approaches are being undertaken to develop technology for understanding the other party in aconversation to the point of identifying a person’s
gender, emotions, and degree of urgency. This can beaccomplished, for example, by detecting whether acustomer is dissatisfied or angry from a call between
that customer and a contact-center operator. This
technology is capable of detecting cold anger (calmand cool expression of anger), which is usually difficult to infer, from not only the loudness and pitch ofvoices but also from conversation rhythm, choice ofwords, etc. It is also able to achieve high-accuracy
recognition of satisfaction, which is a feature that
does not appear as easily as dissatisfaction.
13

Feature Articles

Digitalrepresentationof humans

Approach 1
Emulates human functions
using a computer

Approach 2
Clarifies the physiological
functions of the human brain
and body andtranscribes them

Cognition
Consciousness
MemoryEmotionSociability
Creativity
Physical features
Background information…

Fig. 3. Two main approaches to digital representation of humans.

Speech recognition

1980

1990

Words

Speech synthesis

2000

2003

Assembly-minutes
recognition rate
90%

Utterancerecognition rate
85%
2010

Voice
commands

Clearly
spokensentences

Assembly-minutes
recognition rate
95%

Assembly speeches

Speechrecognition rate
90%

Contact center calls

Naturalness

Utterance bycomputer

2019

Smart-device voice agent

Speechrecognition rate
80%

Clarity

2015

Improving
quality

Diversity

Real-voice
feeling

Achieving diverse
speakers and tones

Understanding emotions and intent
Understanding
satisfaction andfavorability

Understanding
gender andemotions

Fig. 4. Human digitalization—progress in approach using speech technology.

4. Layered structure and hourglass structure
Of importance when studying Digital Twin Computing technology and architecture is whether anhourglass structure can be added to a layered strucNTT Technical Review

Vol. 18 No. 3 Mar. 2020

ture (Fig. 5). This is because the creation of a common layer in the middle of a structured layer can drive
innovation. The Internet is a good example of thisconcept since the positioning of the IP (Internet protocol) layer as a common layer makes for smooth14

Feature Articles

Simulations onearth and outerspace scales

Discovery andresolution of latenturban issues

Creation of sistercities and new
economic activity

Prediction andcontrol of
diseasesMultifaceted
personal
decision-making

Layer×
hourglass…

Application layer

Traffic environments

Urban spaces

Human groups

Digital world presentation layer

Digital twin layer

Cyber/physical interaction layer

ThingsHumansReal space

Expandability improves by creating
an intermediate common layer

Fig. 5. Key to expansion: layered structure × hourglass structure.

interaction between the lower network and upper
application layers. In Digital Twin Computing, aswell, we consider that a narrow section—this common layer—is necessary and that the digital twinlayer in Digital Twin Computing architecture willserve as this section. The digital twin layer maintains
digital twins generated from various types of sensor
data in real space and derivative digital twins generated from computations on digital twins. These digital twins maintained in the digital twin layer serve as
basic constituent elements for constructing diverse
virtual societies.
5. Technology supporting
large-scale computation
It is preferred that the results of discussions among
many human digital twins in cyberspace be fed backto real space and reflected in decision-making. This
will require computing technology that far exceeds
current computer performance. One example of technology supporting such large-scale computation is
the LASOLVTM coherent Ising machine now being
researched and developed at NTT laboratories.
LASOLV is used to find solutions to combinatorial
optimization problems by using special optical pulses
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

to reach a physically stable state through mutualinteraction. We can expect this technology to enablehigh-speed processing on an order of magnitude different from that of current computers. We are also
developing middleware that simplifies the use of
LASOLV in the Python language [3].
6. Use cases of Digital Twin Computing
Digital Twin Computing can be used on a variety of
scales, as shown in Fig. 6. Specifically, the following
uses can be expected.
•	High-speed and parallel debate and decisionmaking by human digital twins•	Development of solutions to difficult national
problems based on the actions of past leaders
who have experience in overcoming crises•	Extensive and detailed urban digitalization
through the integration of digital twins and social
infrastructures such as the transportation system
7. Toward explosion in value throughcomputations on digital twins
Our goal is to make Digital Twin Computing atruly useful concept together with a wide range of15

Feature Articles

Scale ofdigital twins
Person

Group

Town

City

Nation

Globe

Space

DeeperBroader

Exchange and expansion of abilities

Discovery and resolution of urban issues

Earth and outer-space simulations

Incorporate digitalized abilities and extend
perception and physical abilities of realhumans. Convert abilities into content
and distribute.

Discover latent issues in urban living fromconscious/unconscious sentiments ofcitizens. Redevelop cities throughgroup-optimized solutions.

Make future predictions on a global scaleof weather, population, and economic
activities, reproduce urban spaces, andapply to the Sustainable Development
Goals activities andspace development.

Multifaceted personal decision-making

Creation of sister cities

Prediction and control of diseases

Simulate one’s thoughts based on diverse
viewpoints and premises. Support
decision-making based on multifaceted
results.

Create sister cities by combining similar orcontradictory city environments, culturalcharacteristics, and values and build
productive collaborative relationships.

Predict with high accuracy the diffusion of
diseases from the state and behavior ofhuman individuals and minimize thespread of infectious diseases bycontrolling transport,
etc.

CreationControl

TrialsFig. 6. Use cases.

interdisciplinary partners including those in the social
sciences, humanities, etc. We also believe collaboration with a variety of industries is essential to making
Digital Twin Computing a reality. Going forward, weplan to cultivate productive partnerships and promoteresearch and development in this unexplored field
while collecting much knowledge to forge a path to adigital society of the future.
References[1]

Press release issued by NTT, “NTT proposes the ‘Digital Twin Computing Initiative’ – a platform to combine high-precision digital infor-

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

[2]

[3]

mation reflecting the real world to synthesize diverse virtual worlds,
generate novel services and bring about society of the future,” June
10, 2019.
https://www.ntt.co.jp/news2019/1906e/190610a.html
T. Oba, T. Tanaka, and R. Masumura, “Evolution of Speech Recognition System—VoiceRex,” NTT Technical Review, Vol. 17, No. 9, pp.
5–8, Sept. 2019.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201909fa2.html
J. Arai, S. Yagi, H. Uchiyama, K. Tomita, K. Miyahara, T. Tomoe, andK. Horikawa, “LASOLVTM Computing System: Hybrid Platform for
Efficient Combinatorial Optimization,” NTT Technical Review, Vol.
18, No. 1, pp. 35–40, Jan. 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202001fa5.html

16Feature Articles

Ryutaro Kawamura

Senior Vice President, Head of NTT Service Innovation Laboratory Group.
He received a B.S. and M.S. in precision engineering and a Ph.D. in electronics
and information engineering from Hokkaido University in 1987, 1989, and 1996. He
joined NTT Transport Systems Laboratories in 1989. From 1998 to 1999, he was avisiting researcher at Columbia University, USA. From 2003 to 2015, he was amember of the Board of Directors of OSGi Alliance. He is engaged in research on
network reliability techniques, network control and management, high-speed computer networks, active networks, network middleware, and mobile cloud/edge
computing.

NTT Technical Review

Vol. 18 No. 3 Mar. 202017

Feature Articles: NTT R&D Forum 2019 Special
Sessions

Innovative Network for 2030 (Beyond
2020)
Arata Itoh
Senior Vice President, Head of NTT
Information Network Laboratory Group
Abstract

As part of NTT’s Innovative Optical and Wireless Network (IOWN),
we are focusing our research and development efforts on converting thetransport network to be all-photonics based and maximizing its potential.
We are also working on rapidly providing services and optimizing value
chains. This article introduces IOWN initiatives at NTT Information Network Laboratory Group and recent research examples. It is based on a
special session presented by Arata Itoh, senior vice president and head of
NTT Information Network Laboratory Group, at NTT R&D Forum 2019held on November 14 and 15, 2019.
Keywords: IOWN, All-Photonics Network, Cognitive Foundation

1. Issues and solutions toward a smart world
The NTT Group aims to achieve a smart world tosolve social issues in collaboration with business
partners. We have begun to clarify technical issues
and drawn up solutions through trials and verification
experiments with a variety of partners.
Through these efforts, issues in achieving sustainable growth have come to light. The current network
infrastructure is not necessarily equipped to deploy
low-latency services, such as genuinely connectedcars, video delivery, and remote medicine, on anationwide scale, so there is a need to significantly
increase the potential of the future network.
One direction being taken toward solving theseissues is the development of a breakthrough networkthat can transfer inter-terminal Internet protocol (IP)/non-IP traffic without optical-to-electronical signal
conversion. Based on photonics technology, Innovative Optical and Wireless Network (IOWN) will be amajor key technology to solving these issues as it is alarge-capacity, low-latency, flexible, and low-powerNTT Technical Review

Vol. 18 No. 3 Mar. 2020

consumption network.
The following sections introduce the main initiatives in relation to the All-Photonics Network andCognitive Foundation®, which are the fundamental
elements of IOWN.
2. Main elemental technologies in the
All-Photonics Network
Using the wide-band properties and flexibility of
optical communication, a major feature of the AllPhotonics Network is its ability to achieve an optimized function-specific network by providing a fullmesh connection of optical paths in wavelength units
among many locations for each terminal, user, orservice. We established the following four basic functions making up the All-Photonics Network.
(1)	A network transport configuration function
for achieving optical full-mesh networks andwireless access networks to provide highspeed and high-quality end-to-end data transfer(Function 1).
18

Feature Articles

Functions (access local)• Allocates a unique identifier managed in the full-mesh area
to each user in the access area
• Sets logical paths for each identifier

Access area

Photonic GW
Micro
Local
full-mesh area

Photonic
EX

Edge

Functions (local core)• Optical concentrator function (optical concentration
into an optical signal equivalent to a 1 Tbit/s-classoptical TDM signal)• Optical wavelength conversion function

CenterCharacteristics: Massive number of connections. Dynamic pathrequests occur.
Functions: • SO is faster than before; provides logical paths.
• Static low-capacity wavelength paths (10 Gbit/sclass); high-density WDM• Wavelength path shared by multiple logical paths;
optical TDM

Core full-mesh area
Characteristics: Limited and static number of connections
Functions: • Static-wavelength-path setting and wavelength
cross-connect function• Large-capacity wavelength paths (1 Tbit/s class)

SO: service order
TDM: time division multiplexing
WDN: wavelength division multiplexing

Fig. 1. Optical full-mesh network-configuration technologies.

(2)	A network design and control function for
efficiently accommodating the massive number of wavelengths and frequencies needed forconstructing and operating the above networks (Function 2).
(3)	A function-specific network function for optimally combining information and communication technology (ICT) resources, such as
network and computing, and meeting diverse
service requirements (Function 3).
(4)	A terminal function applying photonics-electronics convergence device technology forachieving low power consumption and low
latency per unit amount of data (Function 4).
From these basic functions, we introduce elemental
technologies representative of functions 1–3 (see thearticle “Basic Technologies toward the All-Photonics
Network” in this issue for technologies of Function 4).
2.1 O
 ptical full-mesh network-configuration
technologies (for Function 1)
Two key technologies for achieving an end-to-end
optical full-mesh network are petabit-class largecapacity wavelength-path technology and photonic
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

exchange (EX)/gateway (GW) configuration technology (Fig. 1).
In terms of petabit-class large-capacity wavelengthpath technology, we plan to establish ultra-highspeed optical signal transmission technologies (ultrahigh-speed digital signal processing technology andultra-high-speed optical frontend circuit technology)
for achieving a maximum bit rate of 1 Tbit/s per
wavelength. We will also undertake research anddevelopment of multicore/multimode technology andtechnology for expanding the transmission capacity
per single fiber called transmission-wavelengthrange expansion technology (transmission over theC- and L-bands).
To maintain scalability, we will also research anddevelop photonic EX, a function for directly replacing small-capacity optical paths in the domain-divided local full-mesh area with a large-capacity optical
path in the core full-mesh area, and photonic GW, afunction for turning back signals from the access area
to the local full-mesh area directly in optical form.
These functions achieve low latency and low power
consumption by allocating wavelength paths dynamically instead of performing packet processing.
19

Feature Articles

Smart factory

MaaS

Smart city

(1) Domain-division design method using
wavelength conversion
Reduces scale of problem
Improves efficiency of wavelength accommodation

Network design for IOWN

(2) Topology optimizationpath #1 wavelength 

Improves efficiency of wavelength accommodation

path #2
path #1(3) Multi-rate wavelength-group optimization
path #2 wavelength Interval

Function-specific topology design using geospatial data

LCS/SDK: LASOLVTM

Reduces scale of problem
Improves efficiency of wavelength accommodation
Linking with LASOLVTM and LCS/SDK

Full-mesh-network wavelength design

Computing System/Software Development Kit

Fig. 2. Accommodation-design technology based on spatial data science.

2.2 Accommodation-design technology based on
spatial data science (for Function 2)
As described above, the All-Photonics Network
allocates wavelengths for each terminal, user, or service and enables end-to-end wavelength path connections. In this regard, the explosive increase in thenumber of communication terminals due to the
spread of the Internet of Things, mobility as a service(MaaS), etc. and the resulting combinations of communicating parties means that a massive number of
optical paths will be needed. This need underscores
the importance of accommodation-design technology
for efficiently allocating a limited number of wavelengths (Fig. 2).
There are three key points in accommodationdesign technology.
(1)	Domain division that uses wavelength conversion technology via the photonic EX/
GW functions described above. That is, thedivision of the local area into multiple domains
and performing wavelength conversion at
domain boundaries enables the reuse of thesame wavelength by different domains. This
makes for more efficient use of wavelength
resources. Domain division also reduces thenumber of wavelengths that should be allocated to optical paths. The key to domain division lies in the method of division and optimization of the unit of division such as by making divisions centered about hubs having highdemand for originating and terminating calls.
(2)	Topology optimization. The idea is to derive
optimal paths and topology within a domain
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

and improve the efficiency of wavelength
accommodation while enabling effective provisioning for new or additional installing of
optical fibers.
(3)	Multi-rate wavelength-group optimization.
Grouping optical paths having the same route
within a domain and allocating and managing
wavelengths as a wavelength group make itpossible to increase the efficiency of wavelength accommodation and reduce the scale of
the wavelength-allocation problem. Achieving
an appropriate scale for this problem enables
optimal wavelength allocation using the
LASOLV TM coherent Ising machine and
LASOLV TM Computing System/Software
Development Kit (LCS/SDK)*.
Combining these technologies makes it possible toreduce the number of wavelengths required per fiber
to about 1000, which means that the system is feasible assuming the use of multicore fiber.
2.3 O
 ptical distributed-computing-platform
technology (for Function 3)
We are studying optical distributed processing technology for achieving advanced processing of large
volumes of data from various devices connected to an
optical full-mesh network according to diverse service requirements (Fig. 3).
The key point is the use of the large-capacity andlow-latency properties of an optical full-mesh network*

LCS/SDK: Middleware for solving combinatorial optimization
problems with the LASOLVTM coherent Ising machine.

20Feature Articles

Field-specific dataexchange/analysis

MaaS

…

Energy (VPP)

Optical distributed computing platform
APL
Advanced dataprocessing/analysis functions

CPU
CPU
CPU

APL

CPU
CPU
CPU

CPU APL
CPU
FPGA
Data

Data

+

Data

Data

Ultra-low-latency/
wide-area network
Massive data

GPU
Mem
GPU
Mem APL
GPU
Mem

Data
5G/
Local 5G

Optical access
APL
CPU

Data
Data
Data

Beyond 5G

GPU

Wi-Fi/
DSRC

Data
Data
Data

Regional
BWA
CPU

Position/time

Data
Data
Data

Optical-fiber
environmental
sensing
APL: application
BWA: broadband wireless access
CPU: central processing unit

DSRC: dedicated short-range communications
5G: fifth-generation mobile communications
FPGA: field-programmable gate array

GPU: graphics processing unit
Mem: memory
VPP: virtual power plantFig. 3. Optical distributed-computing-platform technology.

to drive dynamic and high-density cooperative operation of computing resources distributed over a wide
area. Therefore, computing resources located far
from each other can be operated on as if they werelocated at the same location, and ever-changing
resource requests can be flexibly handled. This technology can achieve both highly efficient use of computing resources and advanced data processing.
3. Main elemental technologies in
Cognitive Foundation®
Cognitive Foundation is a mechanism for carryingout integrated deployment, configuring, and linking
as well as management and operation of ICT resources on different layers such as the cloud, network services, and user facilities. In addition to making business more efficient for service providers and driving
their digital transformation, Cognitive Foundation
will enable even higher levels of user conveniencesuch as the rapid provision of services to end users.
Essential technologies for achieving Cognitive Foundation include user-friendly operation technology for
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

automatically designing and autonomously operating
diverse resources and technologies for enhancing andoptimizing the entire system including the wired andwireless systems.
In this area, we are researching and developing
natural wireless-access platform technology as an
optimization technology for wireless resources. The
goal is to achieve optimized wireless areas tailored tousage conditions and to link multiple wireless areas
to enable the provision of seamless wireless connections regardless of the type of wireless connection.
We call such wireless control technology Cradio
including the quality-prediction technology forachieving optimal use of multiple types of wirelessaccess (Fig. 4).
This technology aims to provide comfortable services for users’ various demands by using artificial
intelligence (AI) to predict beforehand dynamicchanges in wireless communications quality accompanying the movement of drones, autonomous driving vehicles, smartphone terminals, etc. and automatically selecting and configuring an optimal wireless environment.
21

Feature Articles

High speed

Low cost

Low latency

Many and varied
services

High reliability

Low power
consumption
Wireless sensing

Requests for diverse and novel wireless access
Combines a variety of wireless systems and achieves wireless access without user
being conscious of the wireless system being used

Predicted
quality

Predicts wireless-access quality beforehand andautomatically selects optimal accessWirelessenvironment
information

Wireless-qualityprediction engine

Optimalselection
Optimalselection

Drone
Smartphone

VehicleFig. 4. Quality-prediction technology for optimal use of multiple types of wireless access.

4. Toward the creation of new value
To enhance the value of optical fiber laid throughout Japan, we are also researching and developing theextensive use of optical fiber beyond communication
fields to non-communication fields.
We can consider, for example, high-efficiency
optical-energy usage technology for maintaining
communications during a disaster by effectively
using infinitesimal amounts of energy in optical fiber

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

transmissions. We can also consider rolling out services by using the optical-fiber environment-monitoring technology that obtains and visualizes environmental data through optical fiber sensing as a regional environmental data platform (Fig. 5).
Our plan is to pursue research and development
toward the creation of new value including noncommunication fields while collaborating with avariety of partners through the IOWN Global Forumand other settings.

22

Feature Articles

Service usage scenarios

Congestion
prediction

Facility
visualization

Conduit-flow
monitoring

Disaster-impact
prediction

Correlation analysis (AI, machine learning, ...)

Database
Optical network

Specialized service knowledge
and technology throughcooperation withexternal parties
Meter

Disaster data

Facility-status data

Road data

Weather data

Services
Independentcompanies/
organizations

Environmental data platform
Broad use of data in a wired area obtained throughdistributed optical-fiber sensing (environmental data)
in services

Fig. 5. Optical-fiber environment-monitoring technology.

Arata Itoh

Senior Vice President, Head of NTT Information Network Laboratory Group.
He received a B.E. and M.E. from Tokyo Institute of Technology in 1986 and1988. Since joining NTT in 1988, he has conducted research and development ofasynchronous transfer mode-based multimedia switching architecture and the Next
Generation Network. He has also been involved with the NTT EAST/WEST R&D
Center and the R&D vision group of the NTT Research and Development Planning
Department.

NTT Technical Review

Vol. 18 No. 3 Mar. 202023

Feature Articles: NTT R&D Forum 2019 Special
Sessions

Basic Technologies toward the
All-Photonics Network
Tetsuomi Sogawa
Senior Vice President, Head of NTT
Science and Core Technology
Laboratory Group
Abstract

Optical technology is important for achieving the All-Photonics Network, a key element of NTT’s Innovative Optical and Wireless Network(IOWN). This article introduces basic research on optical technologies,
including large-capacity optical transmission system/device technology,
photonics-electronics convergence technology, the LASOLVTM coherent
Ising machine, and optical lattice clock network. It is based on a special
session presented by Tetsuomi Sogawa, senior vice president and head of
NTT Science and Core Technology Laboratory Group, at NTT R&D Forum 2019 held on November 14
and 15, 2019.
Keywords: IOWN, All-Photonics Network, photonics-electronics convergence

1. Introduction
Internet traffic continues to dramatically increase,resulting in more power consumed by information
technology equipment called upon to process hugeamounts of data. At the same time, Moore’s Law, thescaling law involving the evolution of large-scale
integrated circuits, is approaching its limit. The miniaturization of semiconductor devices generates leakage currents and heat, thus, degrades performance.
Though the amount of data that must be processed
continues to increase, the inability of scaling up computing power at its traditional pace is becoming afundamental problem.
2. What is the All-Photonics Network?
One target for the All-Photonics Network, a key
element of Innovative Optical and Wireless Network(IOWN), is a 100-fold increase in power efficiency
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

(Fig. 1). To this end, we are studying technology fortransmitting signals using light from the network toterminals and studying new devices such as photonicelectronics convergence devices. We also aim toincrease transmission capacity 125 fold, so we are
studying large-capacity optical transmission system/
device technology using, for example, new opticalfibers such as multicore fiber. Furthermore, to reduce
end-to-end delay by 1/200, we are researching various technologies to, for example, transmit information with no compression when communications do
not allow for delay.
3. Role of increasingly important
optical technology
Easy-to-use electronic technology has been used
for chips that perform a variety of operations in computers. However, the recent trend toward high-density
integration has been accompanied by an increase in24

Feature Articles

Low power consumption

Large capacity, high quality

Low latency

Power efficiency:
100 times higher*1

Transmission capacity:
125 times higher*2

End-to-end delay: 1/200*3

Transmission media• Transmission per wavelength• Wavelengths (optical signal)

Optical fiber cable

Independent

Transmission systems

Light

Light
Light/lightLight (wavelengths) throughout
Wavelengths

Information processing platform
Photonics-electronics convergencedevicesA
B
C
D• No data compression
Wavelength A

Wavelength B*2 Target communication capacity
per fiber

Large-capacity video(uncompressed)
No processing delay

Optical fiber

1000 Tbit/s/fiber*1 Target power efficiency for thephotonics part

• No queueingLight

Speech*3 Target delay for uncompressed video
traffic within the same prefecture in Japan

Fig. 1. Target performance of All-Photonics Network.

the heat generated by wiring within a chip and theresultant constraint on performance. For this reason,
we set as a target a new chip that integrates optics andelectronics with lower power consumption by introducing optical communications technology in the
chip’s wiring section that incorporates high-speed
computing technology characteristics of optical technology. Therefore, the use of optical technology in an
end-to-end manner in conjunction with network opticalization plays an important role in making IOWN areality.
This article focuses on four research themes towardachieving the All-Photonics Network.
4. Large-capacity optical transmission
system/device technology
Compared with the 1980s when optical communications first got underway, communication speeds
using optical fiber have increased by as much as sixorders of magnitude over the last forty years (Fig. 2).
This has been enabled by key inventions such as
semiconductor lasers, planar lightwave circuits, andfiber amplifiers, and transmission capacity has
recently dramatically increased through the development of digital signal processors for digital coherent
communications.
NTT reported in 2019 on a successful experiment
to transmit 1 Tbit/s-per-wavelength optical signals in
35-wavelength division multiplexing at the laboratory level [1]. We also reported on the successful transNTT Technical Review

Vol. 18 No. 3 Mar. 2020

mission of 1 Tbit/s-per-wavelength optical signals at
a practical level for over 1000 km under a commercial
environment using laid optical fiber [2]. The current
outlook is for transmission on the level of 1 Pbit/s per
fiber using, for example, a new fiber structure called
multicore fiber that, as the name implies, incorporates
many cores in a single fiber.
Devices applied to the core, metro, and access networks must evolve steadily, and devices that interconnect datacenters, the demand for which is growing
dramatically, will become increasingly important.
Therefore, there is a need for making as many end-toend connections, including those between datacenters, as possible by optical means, and that is exactly
what we want to achieve with IOWN.
5. Photonics-electronics convergence technology
Although it was very difficult to handle light in thepast, a structure called a photonic crystal (Fig. 3), in
which the refractive index is changed periodically,has made it possible to confine light to a small area
and improve the interaction between light and material. Basic low-power operation of a variety of devices based on photonic crystals, such as optical switches, optical lasers, optical memory, and optical random-access memory, has been demonstrated experimentally.
The roadmap for photonics-electronics convergence technology is shown in Fig. 4. In Step 1, theplan is to achieve a structure that integrates circuits
25

Feature Articles

Transmission capacity per fiber (bit/s)

1P

Digital coherent transmission &high-density
wavelength multiplexing

100T
10T

100G

400 G/ch
100 G/chWavelength multiplexing &optical amplifier

1T

19 Tbit/s

8 Tbit/s

Wavelength filter
(AWG)
ROADM

10G

Electronic time
division multiplexing

GaAs IC

1G

40G-OTN

100G/200G
DSP

Optical amplifier
1980

Semiconductor laser
1990

2000

2010

2020

Commercialization period (year)

AWG: arrayed-waveguide grating
DSP: digital signal processor
GaAs: gallium arsenide

IC: integrated circuit
OTN: optical transport network
ROADM: reconfigurable optical add/drop multiplexer

Fig. 2. Progress in photonics technology enabling traffic growth.

200 nmn-step worth of delay, the form of optical switches
and optical circuit as a logic circuit, which we call an
optical pass gate, can obtain a calculation result
instantaneously with just the time taken to passthrough the optical circuit [3]. Although we are at thebasic evaluation stage with just a small number ofbits, we are also studying the application of opticalpass gates and optical transistors [4].

Fig. 3. Photonic crystal.

6. LASOLVTM coherent Ising machine
mounted using silicon photonics with fibers, analog
integrated circuits (ICs), etc. and achieve high connection speeds with elements outside the chip. In
Step 2, the plan is to directly interconnect chips byshort-range optical wiring, and in Step 3, the plan is
to make inter-core connections within a chip by optical wiring to achieve ultra-low-power consumption.
Step 3 will further improve the performance ofchips by taking advantage of the arithmetic processing that is characteristic of light. While passing
through a typical n-step logic gate would generate
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Next-generation computers, many of which havebeen announced in recent years, are expected to solve
complex problems requiring a huge amount of computational resources that have been difficult withconventional computers [5, 6]. Against this background, we are researching solutions to combinatorial optimization problems—one type of complex
problem—through the use of our coherent Isingmachine called LASOLVTM that uses optical technology, one of NTT’s main focus areas. There are many
combinatorial optimization problems in the realworld. They involve a massive number of combinations
26

Feature Articles

Step 3
Step 2

Optical layer

Ultra-low-power consumption

Step 1

Ultra-high-speed processing
CMOS layer

Control/signalprocessing ASIC
Optical fiberInter-chip opticaltransmission(mm–cm)(1) Intra-chip inter-core opticaltransmission(2) Intra-chip optical signal processing(<1 mm)Analog IC
Silicon photonics chip

Chip peripheral connections:
1.6–3.2 Tbit/s

ASIC: application-specific integrated circuit
CMOS: complementary metal-oxide semiconductor

Fig. 4. Roadmap for photonics-electronics convergence technology.

that are generated when the number of options
increases, a situation that has been difficult with current computers.
Combinatorial optimization problems can be converted to graph problems, and an Ising machine is acomputer based on the new concept of conducting
physical experiments to derive solutions to suchgraph problems. For example, if we were to arrange amagnet oriented to either the S-pole or N-pole at eachnode of a graph according to certain constraints thenrelease our hold on those magnets, the total energy ofthe system would take on its smallest value as themagnets orientate themselves to their most stableconfigurations. This would, in effect, derive ananswer to that graph problem.
LASOLV uses optical technology to create a situation equivalent to this magnet experiment. Specifically, it inputs 2000 optical pulses into an optical fiber
1 km in length and passes them any number of times
through a special optical amplifier to amplify them
repeatedly. The system eventually settles into a state
whereby the phase of each optical pulse is either 0 orπ. If we compare the oscillation of an optical wave tothat of a playground swing, we can say that it settles
into a state whereby the swing is either ahead or
behind. In addition, the machine takes some of the
optical pulses from the optical fiber and measures
their phase states, sets constraints in the graph problem in terms of optical interactions, and returns thesepulses to the optical fiber overlaying them on theoriginal pulses. Repeating this process while circulatNTT Technical Review

Vol. 18 No. 3 Mar. 2020

ing the optical pulses about 1000 times through theoptical fiber provides a solution to the problem(Fig. 5).
On comparing LASOLV with quantum annealingdeveloped by another company, it was found thatLASOLV exhibited superior performance the more
complex the graph structure became, even when solving relatively small problems. It has also been recently found that LASOLV can even be useful in solving
larger-size problems [7].
LASOLV can be applied to complex wavelengthallocation problems in the All-Photonics Network of
IOWN and perform high-load processing in machinelearning.
7. Optical lattice clock network
The optical lattice clock invented by Professor
Hidetoshi Katori of the University of Tokyo loses
only one second every 30 billion years, which is moreaccurate than the advanced cesium atomic clock by
three orders of magnitude. The optical lattice clock
can also be read out by laser, which opens up the possibility of clock transmission over optical fiber. We
are studying what this feature has to offer by constructing a network of optical lattice clocks installedin many of NTT’s central offices.
As suggested by the general theory of relativity,
clocks run faster at higher altitudes, so optical latticeclocks that have achieved 10–18 of total accuracy ofclock frequency allow us to compare remote locations
27

Feature Articles

PSA based on PPLN

“Networked” OPOs oscillates at phase
configuration that best stabilizes the wholenetwork.
Solution to the given
Ising model problem
Spin-spin interaction:
Measurement-feedback

1-km fiber cavity
Spins:
Optical parametric oscillator phases(a) LASOLV operation concept
FPGA: field-programmable gate away
OPO: optical parametric oscillator(b) LASOLV external view

PPLN: periodically poled lithium niobate
PSA: phase sensitive amplifierFig. 5. LASOLVTM.

and measure differences in altitude as small as 1 cm.
In Japan, an earthquake-prone country, this meansthat minute crustal movements and massive gravitational movements can be detected, which points tothe possibility of constructing a very effective, safe,
and secure infrastructures.
However, when carrying such ultra-high-accuracy
signals by optical fiber, it is necessary to control
drops in accuracy caused by optical fiber fluctuations. We are currently conducting a trial of an opticallattice clock network that connects various bases by
NTT EAST optical fiber and includes relay equipment installed in NTT central offices (relay stations)to cancel such optical fiber fluctuations while maintaining high accuracy (Fig. 6). In the future, wewould like to see an optical lattice clock network
introduced into IOWN for developing new types of

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

time-related business.
8. Toward Achieving the All-Photonics Network
Going forward, we will promote research anddevelopment efforts to achieve world-best, worldfirst, and impressive technology. We will promotefurther research and development of photonics-electronics convergence technology, the key to achieving
the All-Photonics Network.
We also plan to make IOWN a reality in collaboration with experts in a wide array of research andtechnology fields while holding discussions withvarious companies and universities via the IOWN
Global Forum [8]. To all parties involved in thiseffort, we look forward to your support and cooperation.

28

Feature Articles

RIKEN

Atsugi-Hongo-Wako
Fiber distance: 150 km

The Universityof Tokyo

Wako
Hongo
Relay station 4

Ultra-high-accuracy opticalfrequency transmission
equipment (repeater)

Relay station 2
Laser
NTT

Relay station 3
Atsugi

Relay station 1

Chamber
Transportable opticallattice clock
NTT Unit No. 1

Collaborating with the University of Tokyo, RIKEN, andNTT and establishing fiber delivery technology with theworld’s most accurate optical frequency standard@ NTT GEOSPACE

Fig. 6. Optical lattice clock network trial.

References
[1]
[2]

[3]

Press release issued by NTT on Mar. 7, 2019 (in Japanese), https://www.ntt.co.jp/news2019/1903/190307a.html
Press release issued by NTT, “Long-distance Transmission of 1 Tbit/s
Ultra-high-capacity Optical Signal—World record 1,122 km transmission in a commercial environment,” June 19, 2019, https://www.
ntt.co.jp/news2019/1906e/190619a.html
A. Shinya, T. Ishihara, K. Inoue, K. Nozaki, and M. Notomi,“Ultralow-latency Optical Circuit Based on Optical Pass Gate Logic,”
NTT Technical Review, Vol. 16, No. 7, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201807fa6.html

[4]
[5]
[6]
[7]
[8]

Press release issued by NTT on Apr. 16, 2019 (in Japanese), https://www.ntt.co.jp/news2019/1904/190416a.html
Google AI Quantum, https://ai.google/research/teams/applied-science/quantum/
D-Wave, https://www.dwavesys.com/
Press release issued by NTT on May 25, 2019 (in Japanese), https://www.ntt.co.jp/news2019/1905/190525a.html
IOWN Global Forum, https://iowngf.org/

Trademark notes

All brand, product, and company/organization names that appear in thisarticle are trademarks or registered trademarks of their respective owners.

Tetsuomi Sogawa

Senior Vice President, Head of NTT Science and Core Technology Laboratory
Group.
He received a B.S., M.S., and Ph.D. in electrical engineering from the Universityof Tokyo in 1986, 1988, and 1991. He joined NTT Basic Research Laboratories in1991. From 1999 to 2000, he was a guest scientist at Paul Drude Institute in Berlin,
Germany, where he investigated acoustic spin transport phenomena in semiconductor quantum structures. From 2004 to 2006, he worked for the Council for Scienceand Technology Policy, Cabinet Office, Japan, as a deputy director for policy planning. His research interests include fabrication technology of low-dimensional
nanostructures, optical properties of quantum dots/wires and photonic crystals, spinrelated phenomena, and surface acoustic wave application to nanostructures.

NTT Technical Review

Vol. 18 No. 3 Mar. 202029

Feature Articles: NTT R&D Forum 2019 Special
Sessions

Upgrade Reality—Reality in IOWN
Concept
Kazuhiro Gomi
President & CEO, NTT Research, Inc.
Abstract

This article introduces the vision of NTT Research, Inc., which began
its operations in July 2019 with the three laboratories. This article is basedon the special session for NTT Research, Inc. at the NTT R&D Forum2019, which was held on November 14 and 15, 2019.
Keywords: advanced basic research, IOWN, Upgrade Reality

1. The current situation of ICT sector andlaunch of NTT Research, Inc.
Looking at the recent trends in the information andcommunication technology (ICT) market, we see
new innovations making good progress in the areas ofInternet of Things (IoT), big data, artificial intelligence (AI), and automation. Many technologies tosupport such trends are now affordably available,
which triggered digital transformation (DX) in manybusiness sectors, which has created tremendous positive impact on the way we do business and live ourlives. However, there are several limiting factors on
the horizon that could impede this evolution. These
factors are ICT energy consumption, saturated
Moore’s Law, and security (or privacy) (Fig. 1).
To sustain DX for over a long period, we see theneed of having some kind of “game changer,” rather
than incremental improvements. That is exactly how
NTT research and development (R&D) laboratories
view the Innovative Optical and Wireless Network(IOWN) initiative. IOWN is a reflection of NTT
R&D’s focus on long-term solutions.
NTT Research, Inc. was founded in July 2019 to
conduct basic research activities related to the key
components of the IOWN initiative. This company isbased in the area outside of San Francisco, California,
NTT Technical Review

Vol. 18 No. 3 Mar. 2020in the United States known as Silicon Valley. The key
advantage of this setting is this is where many of the
major global talent and global research partners arelocated; therefore, collaborations with them are easily facilitated. Also, by having this footprint in Silicon
Valley, NTT R&D as a whole becomes a part of such
communities. The vision of NTT Research, Inc. is tofurther enhance the research activities that NTT has
been engaged in for a long time through such collaborations (Fig. 2).
2. The activities of each laboratory
NTT Research, Inc. is currently organized into
three research areas, as shown in Fig. 3. These are thePhysics & Informatics Laboratories (NTT PHI Labs),
the Cryptography & Information Security Laboratories (NTT CIS Labs), and the Medical & Health
Informatics Laboratories (NTT MEI Labs).
2.1 NTT PHI Labs
PHI Labs studies completely novel optical computers called coherent Ising machines (CIMs), which aredesigned to solve combinatorial optimization problems efficiently. An example of a combinatorial optimization problem is the traveling salesman problem,
which is to find the shortest overall route covering all30

Feature Articles

Technology trends
supporting DX

Limiting factors• Energy consumption• Moore’s Law limitations• Security• Privacy• IoT• Big data
• AI
• AutomationGame change
IOWN

Fig. 1. Technology trends and limiting factors.

GlobaltalentNTT R&D
laboratories
Globalresearch partners

NTT Research, Inc.
Silicon Valley, California

Fig. 2. Why NTT Research, Inc. was established in Silicon Valley.

Physics and Informatics (PHI Labs)
Cryptography and Information Security (CIS Labs)
Medical and Health Informatics (MEI Labs)

Fig. 3. Laboratories in NTT Research, Inc.

the customers. The number of possible routes
increases exponentially as the numbers of customers
and possible paths increase. This problem can takeyears to solve on conventional computers when dealing with practical cases. The concept of the travelingsalesman problem can be applied in many similar
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

real-life problems such as optimal traffic control tominimize congestion in a large city and wavelength
allocation in mobile phone services.
Over the years, NTT R&D has been working on
optical versions of the CIM, which combine the principles of quantum information processing with thoseof brain information processing. NTT Research, Inc.
announced in November the start of a research collaboration with eight external partners in the UnitedStates, Canada, and Australia (five US-based universities, one Australian university, NASA, and one
software startup) to enhance the research activities
around CIMs. Together with these partners, PHI Labs
studies the theoretical aspects of these machines,
conducts related experiments, and explores applications for which these machines would solve their
31

Feature Articles

issues effectively and efficiently.
2.2 NTT CIS LabsCIS Labs conducts research on cryptography. To
build a smart world and connected society, we need tobe ever-more attentive on the issues around privacy
and security. Crypto systems are developed to protect
security and privacy; however, the implementation ofsuch systems tends to make information sharing moredifficult. In other words, easy information sharing
and protection of privacy/security are trade-offs inmany cases. CIS Labs is researching new cryptographic techniques that enhance the flexibility in
information exchange while maintaining security andprivacy. Another topic that this group is tackling is
post-quantum cryptography, addressing the concernthat most current crypto systems would be broken
when quantum computers are in place.
CIS Labs is also working on the topic of blockchain
security. Blockchain has been implemented in several
applications such as crypto currencies and fintech
applications. However, there has not been muchresearch into its security implications and scalability
issues. One of CIS Labs’ focus areas is the proofs ofblockchain security, for example, to answer the question, “under what conditions can a blockchain be
broken.”We will continue to assemble a dream team of top
experts and talent. Currently, the team consists ofresearchers and scientists who came from the faculties of top level universities in this field such as the
University of Texas, Northeastern University, Princeton University, and Cornell University.
2.3 NTT MEI Labs
MEI Labs’ field of research is medical and healthsciences. This is closely related to Digital Twin Computing (DTC), an important component of the IOWN
initiative. Medical practice is always looking for thebest solution along with the next best. With this inmind, to have your digital twin (alter ego) in the cyberworld and administer treatments to it would create ahuge advantage for medical care in the future. If aparticular treatment, tested on the digital twin, does
not work, we can try another approach without risking the patient’s health. Furthermore, if you have aprecisely simulated model of your body or part ofyour body (i.e. organs), it would create a revolution in
the world of medical practice. Today, with significant
progress in bio-sensing technologies, a vast amountof medical information is becoming available.
Together with the innovations in AI, big data, and
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

robotics, precise human-body simulation and medical
treatment based on it will be the next step we need topursue. In the past, medical records, and even test andimaging results, existed solely within the handwritten
analog world, but now these data are all digitally
stored. We envision that, by leveraging the output
from advanced crypto technologies, such data will beeffectively exchanged among bio-medical/data-analytic experts to study and create simulated humanbody models while maintaining patients’ privacy.
Many experts have been pointing out that medical
and health sciences need to collaborate in a morerobust manner with math, science, and engineering
disciplines. In this new era where information technologies are progressing rapidly, we can expect
medical and health sciences to leverage a variety ofthese innovations. However, such an approach should
be ethically sound. The fundamentals of human community needs to be preserved while developing cutting edge medical and life-science technologies. Oneof the conditions for achieving this is the integrity ofthe members engaged in the research activities.
In Silicon Valley, such collaborations are normal
between industries, academia, medical facilities, andstartups, each of which brings its own strengths. We
see many young people in their 20s and 30s proving
their mettle in the competitive environment of Silicon
Valley. MEI Labs offers an attractive workplace forhighly motivated researchers in the fields of medicine, data science, and material science.
3. Researchers at NTT Research, Inc.
We have recruited over 20 researchers and scientists thus far, all of whom have doctorates and 10 holdprofessorships. They have variety of backgrounds interms of where they are from. Just under ten are from
North America, one is from Europe, and just under
ten are from Asia.
We also formalized numerous research collaborations, mostly with universities in the United States(Fig. 4). These partnerships effectively enhance thetalent and resources engaged in our research activates, expand the horizon of our thinking process, andstrengthen our ties to research networks. We are envisioning to further enhance these partnerships moving
forward.
These basic research NTT Research, Inc. isengaged in is in line with the IOWN initiative. Our
activities are also to support further development ofNTT’s global businesses from innovation perspectives
along with two other companies, NTT Disruption and32

Feature Articles

Universities• California Institute of Technology• Cornell University• Massachusetts Institute of Technology• Princeton University• Rutgers University• Stanford University• Swinburne University of Technology (Australia)• Technical University of Munich (Germany)• University of California, Berkeley• University of California, Los Angeles
Other research institutions• NASA Ames Research Center in Silicon Valley• 1Qbit (Canada)Fig. 4. Research partners.

NTT Venture Capital. NTT Disruption will be showcasing what can be done in the near future using thelatest DX technologies. It will unearth concrete solutions tailored to various verticals such as healthcare,financial, autonomous driving, sports, and entertainment. NTT Venture Capital will take responsibility
for the medium term by identifying and investing inpromising startups. NTT Research, Inc. is ideallyplaced to take charge in addressing longer-term innovation by envisioning “beyond Internet,” which the

IOWN initiative is targeting.
4. Conclusion
The motto of NTT Research, Inc. is Upgrade Reality. By “reality” we mean so-called accepted norms
that define what we think of as the real world. Our
focus will always be on fundamental improvements
to the status quo.

Kazuhiro Gomi

President & CEO, NTT Research, Inc.
He joined NTT in 1985. Before taking up his current post in April 2019, he served
as vice president (VP) of the Global Business Department of NTT Communications
from 2001 to 2004, after which he served as VP of the Global IP Network Business
Unit of NTT America (2004 to 2009), chief operating officer of NTT America (2009to 2010), and president and chief executive officer of NTT America (2010 to 2019).

NTT Technical Review

Vol. 18 No. 3 Mar. 202033

Feature Articles: ICT R&D Initiatives for a Smart World
that Harmonizes People and Environments

R&D Initiatives to Make the Most of
Human Abilities by Amplifying
Human Intelligence to Achieve a
Smart World that Harmonizes People
and Environments
Akihito Akutsu, Kenichi Minami, Hiroaki Kawata,
and Shigekuni Kondo
Abstract

Information and communication technology (ICT) will soon be necessary to make the most of andsupplement people’s abilities and support their activities by harmonizing people and environments
through natural interactions that seamlessly blend with their activities. This article introduces the ICT
research and development initiatives at NTT Service Evolution Laboratories to achieve a smart worldthat harmonizes people and environments.
Keywords: Point of Atmosphere (PoA), ambient, activity support

1. Introduction
Thanks to the remarkable progress in informationand communication technology (ICT), it is now possible to access the information one needs from asmartphone or personal computer anytime and anywhere, and Internet services, such as social networking services, online shops, and navigation services,such as maps and route guidance, have become indispensable. Current Internet services are mainly provided via web browsers, and many more users cantake advantage of these benefits not only throughimproved content attractiveness but also by improving ease-of-use (usability, web accessibility, findability) and operation (interoperability). However, services currently provided on the Internet are based on
the premise that users access information with therequired knowledge and a specific purpose. Therefore, a corresponding level of ICT literacy is required
to take advantage of the benefits of such services.
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Achieving a world in which everyone can benefit
from ICT services regardless of their literacy or situation will entail not only users actively accessing
information but also require the environments surrounding users to ‘understand’ them and provide
information in a form that people can recognize
within the scope of their abilities without being burdened. One approach to this challenge is ambient
computing [1]. Ambient means “peripheral,” “environmental,” or “surrounding.” Thus, ambient computing is a concept of a world in which computersgather required information on behalf of people without anyone needing to perform operations or commands. With the recent developments in the Internetof Things and artificial intelligence, it has become
possible to collect a large amount of information
about people and the environment, enabling presentation of necessary information based on the results ofpredictions made by recognizing people’s activity
patterns up to the present. We believe that the advent
34

Feature Articles

Now

Goal

Web

Virtual world

Communications and interaction
via the web

Communications and interactions
via virtual world

The world achieved

The world accessedInternet (web)Main information
presented

Visual and auditory-based
information

Features,requirements

Active information access
Usability, accessibility
Findability
Interoperability

A virtual world that processes
information and precisely reproduces
the real world
Information based on the values,
etc. of people that cannot beperceived with the five senses orhuman cognitive abilities
Passive information access
Diversity
Ambient intelligenceFig. 1. The target world of PoA.

of a smart society where the environment can ‘understand’ people and provide them with the information
they need and make proposals to them will be possible in the near future. However, the presentation of
information and proposals should not impede humanactivities or cause stress. We believe ICT will be necessary to make the most of and supplement people’sabilities and support their activities through naturalinteractions that seamlessly blend with their activities. These Feature Articles introduce the ICT
research and development (R&D) initiatives of NTT
Service Evolution Laboratories to achieve a smart
world that harmonizes people and environments[2–5].
2. Point of Atmosphere (PoA)
NTT announced the establishment of the Point ofAtmosphere (PoA) project at the NTT R&D Forum
held in autumn 2018 [6]. This project aims to achievea smart world that enables natural interactions byharmonizing people and environments. The surrounding environment can support people in making
decisions by enabling the environments to ‘understand’ people’s actions, intentions, and feelings invarious situations without them being aware of theindividual devices around them. For example, various
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

ICT devices around the house could be linked withinformation obtained from people and the environment to create an illusion of a raincoat hanging on thewall or a wet floor to naturally convey the fact that itis going to rain today. To achieve this world, NTT
Service Evolution Laboratories has defined PoA as
any interface that seamlessly connects the real andvirtual worlds and meets the following requirements:•	It must collect real-world information about
people and environments in such a way that thesensing target is not burdened.
•	It must process the collected information toaccurately reproduce real-world people and environments in the virtual world.
•	It must provide people with new perspective in anatural manner.
PoA is not limited to just being a user interface but
is also an interface with a wider meaning that includes
information processing for people and environments.
To date, information presented via the Internet has
been mainly based on audiovisual information andaccessed purposely by people. PoA also uses information that cannot be obtained with the five senses,
such as values, and enables people to naturally recognize information from the surrounding environment(Fig. 1).
The aim of PoA is to use various information
35

Feature Articles

Virtual world

Understanding environments and people,
including their inner senses
Converting information so everyone can receive it easily

Point of Atmosphere (PoA)

Real world

Activity support
Using wide-ranging information collected in the realworld to predict the future, present information
matched to the person, and broaden the range ofhuman activities.

Broadening the range of activities
Making the boundary between the real and virtualworlds seamless, enabling real world-like activities
in the virtual world, and expressing virtual images
in the real world as if they were real.

Fig. 2. Linking virtual and real worlds via PoA.

collected from the real world for virtual-world processing to predict the future or enable people to naturally recognize the information that they need andfeed the information back to them. Enabling realworld-like behaviors in the virtual world in this way
makes it possible to make the most of and supplementhuman abilities and support voluntary improvement
in behavior, thus, expanding the range of humanactivities (Fig. 2).
3. Use cases
3.1 Activity support by predicting the future
In healthcare, PoA could be used to prompt seemingly healthy people to pay more attention to theirhealth by projecting a haggard future version ofthemselves. Conveying such useful information fromvarious surrounding objects (e.g. display devices,
phones, speakers, and lights) will enable people tolead healthier lives. In transportation, this technology
could enable people to obtain train departure times
without the need to look at a timetable or prevent
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

being hit by a car by being provided a sense of a car
approaching. Collecting large amounts of information from the real world will make it possible to use
the virtual world to predict the occurrence of unforeseeable circumstances such as catastrophic disasters,
thus, minimizing the damage in the real world.
3.2 S
 upporting activities that transcend time andspace
When someone cannot attend a sporting event at an
actual venue due to physical limitations, it will bepossible through PoA for that person to watch theevent from home but still be able to engage in realcommunications with the spectators as if he/she wereactually there at the event. One could also have theexperience of viewing a sporting event and feelingthe hardships and joys of the players as if one were aplayer or manager. In addition, by transcending timeand space, one could enjoy games between famous
players of the past and present.

36

Feature Articles

4. Supporting technologies

5. Conclusion

Enabling PoA will require technologies to sense
both people and environments, transmit and process
information obtained by sensing, and feed information back to people and the environment. As well as
information about the five senses, physiologicalinformation, behaviors, and surrounding environments, technology for sensing people and environments will also collect information that cannot beperceived either directly or from the surrounding
environment without burdening people to naturally
supplement and support human activities. The technology for processing and transmitting information
obtained by sensing will process the information collected in the virtual world, achieve zero latency media
transmission to present the information to people sothat they would not feel transmission and processing
delays, and achieve structuring of people’s innersenses such as their emotions, values, and thoughts
by elucidating mechanisms of their behavior andthoughts. This will make it possible to communicate
and interact in ways that transcend time and space or
to guide people to live a better life by appealing totheir instincts. Technology to provide feedback toenvironments and people will detect information thatmakes the most of the five senses and provide information that cannot normally be sensed in a form thatis recognizable in order to expand the range of humanactivities. As the environments surrounding people
become smarter, they will ‘understand’ people’sabilities and situations and interact with them as necessary or enable communication in which the boundaries between real space and virtual space are imperceptible, thus, helping people obtain abilities they did
not originally have or lost. These technologies willprocess the collected human and environmental
information in virtual space to precisely reproduce
and expand the real world, offer support for realworld human activity, and make the most of andsupplement abilities.

This article described initiatives to achieve a smart
world where people and the environment are in harmony. For more people to benefit from ICT and to
expand the range of their activities by making themost of their abilities, it is necessary for such technology to ‘understand’ how people perceive information,
how it should be fed back to them, and how they cannaturally recognize information as they need it atcertain times and in certain situations, but not from a
conventional viewpoint of how to present information. NTT Service Evolution Laboratories is promoting R&D of technologies that will contribute to suchhuman-centered communication and interaction services.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

References
[1]
[2]

[3]

[4]

[5]

[6]

M. Murata, “Towards Ambient Information Society,” J. IEICE, Vo. 93,
No. 3, pp. 233–238, 2010.
A. Sagata, T. Sano, K. Hidaka, T. Satou, S. Fukatsu, T. Mukouchi, and
H. Nagata, “Natural Communication Technologies Providing Information through Natural Exchanges that Do Not Interfere with Human
Activities,” NTT Technical Review, Vol. 18, No. 3, pp. 39–45, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202003fa6.html
H. Seshimo, Y. Nishikawa, T. Kurahashi, Y. Koike, O. Matsuda, S.
Eitoku, T. Watanabe, T. Sato, T. Isezaki, and M. Sasagawa, “Ambient
Assistance Technology Supporting Human Behavior by Understanding the Relationship between People and Environments,” NTT Technical Review, Vol. 18, No. 3, pp. 46–51, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202003fa7.html
Y. Matsuura, H. Toda, M. Nakatani, T. Kurashima, and H. Saito, “User
Experience Method for Deeper Understanding People’s Values to
Design and Change Their Behavior,” NTT Technical Review, Vol. 18,
No. 3, pp. 52–58, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202003fa8.html
A. Hayashi, J. Watanabe, and K. Shimizu, “Sports Social View—
Method of Inclusive Sports Appreciation Extracting and Sharing the
Essence of Sports through the Sense of Touch,” NTT Technical
Review, Vol. 18, No. 3, pp. 59–64, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202003fa9.html
Press release issued by NTT on Nov. 28, 2018 (in Japanese).
https://www.ntt.co.jp/news2018/1811/181128b.html

37

Feature Articles

Akihito Akutsu

Vice President, NTT Service Evolution Laboratories.
He received an M.E. in engineering from Chiba
University in 1990 and a Ph.D. in natural scienceand technology from Kanazawa University in
2001. Since joining NTT in 1990, he has been
engaged in R&D of video indexing technology
based on image/video processing, and manmachine interface architecture design. From2003 to 2006, he was with NTT EAST, where he
was involved in managing a joint venture
between NTT EAST and Japanese broadcasters.
In 2008, he was appointed Director of NTT
Cyber Solutions Laboratories (now NTT Service
Evolution Laboratories), where he worked on an
R&D project focused on broadband and broadcast services. In October 2013, he was appointed
Executive Producer of 4K/8K HEVC (High Efficiency Video Coding) at NTT Media Intelligence
Laboratories. He received the Young Engineer
Award and Best Paper Award from the Instituteof Electronics, Information and Communication
Engineers (IEICE) in 1993 and 2000, respectively. He is a member of IEICE.

Hiroaki Kawata

Senior Research Engineer, Co-Innovation Promotion Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in electrical engineering from Hokkaido University in 2001 and2003 and joined NTT in 2003. His research interests include remote collaboration system, userinterfaces, and telepresence technologies. He is amember of IEICE.

Shigekuni Kondo

Senior Research Engineer, Planning Section,
NTT Service Evolution Laboratories.
He received a B.E. in applied physics and an
M.S. in integrated design engineering from Keio
University, Kanagawa in 2003 and 2005 andjoined NTT in 2005. His research interests
include software modularity, home networks,
and user interfaces. He is a member of IEICE.

Kenichi Minami

Director of Research, Planning Section, NTT
Service Evolution Laboratories.
He received a B.E. in electronic engineeringand an M.S. in biomedical engineering from
Keio University, Kanagawa in 1991 and 1993. Hereceived an MBA from Thunderbird, Global
School of Management, Arizona, USA, in 2002.
He has been engaged in R&D management in thedevelopment of “Kirari!” immersive telepresence technology since 2016. From 2012 to 2014,
he was responsible for the development ofmobile application services at NTT DOCOMO.
His research interests include image and audio
processing, user interfaces, and telepresencetechnologies. He is a member of IEICE.

NTT Technical Review

Vol. 18 No. 3 Mar. 202038

Feature Articles: ICT R&D Initiatives for a Smart World
that Harmonizes People and Environments

Natural Communication Technologies
Providing Information throughNatural Exchanges that Do Not
Interfere with Human Activities
Atsushi Sagata, Takashi Sano, Kota Hidaka,
Takashi Satou, Shinji Fukatsu, Takafumi Mukouchi,
and Hidenobu Nagata
Abstract

NTT Service Evolution Laboratories is conducting research on providing high-presence viewing experiences involving virtual people and objects as if they have been brought to life right in front of the viewers. This article introduces natural communication technologies providing natural and valuable user
experiences by creating people, objects, and space that emulate an incredibly realistic presence andseamlessly linking them to the real world through natural interactions without any sense of latency.
Keywords: natural experience, space and object creation, zero latency

1. Towards creating natural experiences throughthe intersection of real and virtual space
We have been focusing on transmitting informationof people and objects from remote locations to fully
reproduce them in other remote locations to make it
feel as if they are right in front of us [1, 2]. Now thathighly intelligent applications and services have permeated people’s lives and virtual reality (VR) andaugmented reality (AR) have become more accessible, we would like to further expand the range ofexperiences offered to users and offer and have thembe more natural. Our aim is not only to faithfully
reproduce information of people and objects at alocation but also create objects through video projection and audio that give the feeling that they havebeen brought to life, thus create valuable experiencesthat transcend reality. We are studying what aspects
are important for providing such experiences. Whileaccurately expressing objects is valuable in terms ofhigher reality, transforming or exaggerating them
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

may result in expressions with more impact. For
example, in the “Great Wave off Kanagawa” from the“Thirty-six Views of Mount Fuji,” Katsushika Hokusai was able to express the wave as a still image but
as if it were moving. Perhaps some people feel thepower of this wave more than objects in realist paintings or photographs due to Hokusai’s unique expression that gives it movement. Thus, if someone’s
understandings of a large wave is much more powerful than that of a wave seen in a photograph, Hokusai’s painting can be said to have created a real experience for him or her that transcends reality.
1.1 Society 5.0
The advent of the information society has made it
possible to digitize various objects in real space andenable complex re-construction and expression incyberspace. Continuing from the hunting/gatheringsociety (Society 1.0), agricultural society (Society2.0), industrial society (Society 3.0) and informationsociety (Society 4.0), the Cabinet Office of Japan
39

Feature Articles

defines Society 5.0 as “a human-centered society thatbalances economic advancement with the resolutionof social problems by a system that highly integrates
cyberspace and physical space” [3]. In other words, inaddition to accurately imitating real objects that exist
in real space, there is an expectation that reality willbe transcended through the fusion of the real and virtual.
1.2 Generative adversarial networks
Generative adversarial networks (GANs) are key
for creating video that allows viewers to perceive an
object that looks real or expressing objects that transcend the original [4]. Through this technology, computer graphics (CG) images with the quality of real
photos is becoming a reality. It is also becoming possible using GANs to create completely different
images but with the same characteristics of the reference images. GANs enable processing of tasks, suchas sensing the actors on stage, then generating imagesof animals having the same characteristics of theactors’ movements. For example, in the kabuki dance
performance “Renjishi” (Two Lions), there is a scene
where the affection between the parent and child isexpressed through the performance of a lion watchingover its cub as it runs up the side of a valley. Using
GANs, we even can entertain audiences by generating realistic images of the lions as having the samecharacteristics as the actors. It would be effective torealistically express the kabuki lion as a “lion,” but if
it were possible to create an experience of Utagawa
Hiroshige’s woodblock print “shishi no kootoshi” (alion drops its cub) that seemed real in front of viewer’s eyes, it could be considered as an experience that
surpassed reality. When presenting an object generated with image-processing technology, to makeviewers feel as if life has been breathed into it, it isimportant for the viewers to feel that the generated
object is not just a mere imitation. If it is recognized
as a mere imitation, it will be difficult to stir people’semotions. Although it is a major challenge, technology is required to achieve a viewer experience inwhich generated objects appear to move autonomously. Research on predicting human postures
based on the skeleton has begun, and it is somewhat
possible to predict future postures based on prelearned motions and reproducible motions that havesome form. We aim to achieve natural exchanges inwhich the object/person that is sensed moves naturally and autonomously, so that in its interactions
with, for example, actual people, there are no distractions due to processing latency, etc.
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

1.3 Natural communication technologies
To achieve these objectives, we have been researching and developing natural communication technologies. Such technologies consist of the following fiveelements; (1) space and object creation technology tofreely create seemingly real space and objects of thehuman imagination that transcend reality, (2) zerolatency media technology that both reduces physicaldelays in transmission and processing, thus eliminating sensory delays such as the discomfort people feel
due to latency, (3) 2D/3D video display technologythat enables both two-dimensional (2D) and threedimensional (3D) displays to be viewed naturally, (4)information-presentation technology to enable natural interactions between reality and virtual space, and(5) new approaches collectively called “five senses +X transmission technology” for transmitting and presenting not only our five senses but also one’s psychological feelings directly and naturally.
2. Space and object creation technology
Space and object creation technology estimatesdata beyond the obtained sensing data by taking into
account the past forms of the same scene or person.
We have thus far developed a technology to generate
3D spatial information (CG model) from 2D video inreal time using deep learning (Fig. 1). This technology was actually used in a kabuki performance called“Cho Kabuki,” which is a new type of kabuki performance using information and communication technology. It was held during the Niconico Chokaigi
2019 festival organized by video streaming company
Dwango Co., Ltd [5]. Although 3D information is notincluded in 2D video, this technology creates 3D
information based on similar past scenes in real time
and transforms the characters that appear in Cho
Kabuki into other CG characters, enabling a new yet
characteristically Cho Kabuki production. In the
future, we plan to apply transformation to other liveaction video by combining 3D spatial information(CG models) and object-extraction technology.
3. Zero latency media technology
Physical delays in transmission and processing is amajor issue in achieving natural interaction at remotelocations by VR/AR. Although efforts have been
made to reduce these physical delays and commercialization has advanced to some extent, it is physically impossible to reduce delay to zero, even at light
speed. Thus, to achieve natural interaction, we
40

Feature Articles

Camera image

CG character

Fig. 1. Real-time 3D information synthesis.

believe that technology is needed that not only eliminates physical delays but also eliminates the discomfort that humans feel from delays as well as eliminating sensory delay. Thus, we are researching zerolatency media technology. Specifically, this ongoing
research involves clarifying the mechanisms of sensory delay from various information such as peripheral situations and behavior patterns, creating morenatural prediction technology that does not cause
discomfort due to delay, and clarifying the prediction
mechanisms in the brain to eliminate sensory delay
due to the world predicted in the brain.
4. 2D/3D video display technology
NTT has developed “HiddenStereo”—a stereovideo-generation technology that uses the characteristics of human vision to enable users to enjoy 3D
images when wearing stereo glasses and clear 2D
images without stereo glasses [6]. This technology
generates left and right images by adding and subtracting a disparity inducer to a 2D image to generate
left and right parallax. When the disparity-inducer
components are combined, they cancel each other out
so that only the original 2D image is visible whenviewed with the naked eye. Depth information of the
2D image is required to generate the disparity-inducer components. In the case of stereo images, depth
information can be obtained using epipolar geometry,
etc., but a great deal of operations and ingenuity are
required to capture the precise depth information.
With normal 2D images taken with a monocular camera, however, the depth information of each pixel
cannot be obtained. We are engaged in automatic
generation and systemization of “HiddenStereo” thatentails depth estimation using deep learning models,
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

extraction of objects through frame and backgrounddifferences, and instance segmentation using deeplearning models for 2D images taken with monocularcameras.
5. Information-presentation technology fornatural interactions
To enable natural and realistic information-presentation technologies, we are conducting research anddevelopment on 360-degree glasses-free tabletop 3D
display technology (360° autostereoscopic 3D)(Fig. 2) and sound field synthesis technology. This
technology enables viewing of 3D objects with binocular disparity on a screen on a table without using
3D glasses by combining multiple projectors arranged
in a circle and a special screen called a spatiallyimaged iris plane screen [7]. A large 120 cm-diameter
screen and optical linear blending enable smooth
movement of the point of view (smooth motion
parallax), even though the number of projectors is
1/4 to 1/10 that of conventional technology. Sound
field synthesis is a technology that can reproduce
a sound field using a linear loudspeaker array ofmultiple loudspeakers in a line. This technology cancontrol the distance between a sound source and audience member as well as its direction. In past eventsand installations, we used this technology to produce
virtual sound sources that come close to audience
members to reproduce kabuki performances [8] andgoalball matches, a team sport played by visually
impaired athletes. We have been investigating an
extension of this technology to reproduce sound in alimited area. We have succeeded in reproducing asound field using a multipole-loudspeaker array consisting of multiple small speakers closely arranged in41

Feature Articles

Multiple projectors

3D object

4K high-quality 3DSpecial optical screen
Downsized
simple 3D

Horizontal
viewpoint
Adjacent viewpoint images are blended optically.

Retinalimage

Perception
image

Single edge

SparseLuminance
ratio
10 : 0
9:1
8:2
7:3
6:4
5:5
4:6
3:7
2:8
1:9
0 : 10

Double edge

Luminance
distribution

Projected images

Visual perception of blended images
with different luminance ratio

Fig. 2. 360-degree glasses-free tabletop 3D display technology (360˚ autostereoscopic 3D).

a Cartesian grid. This array enables control of thedirectivity pattern of an arbitrary sound source while
being smaller than current speaker arrays [9, 10].
These technologies place importance on achieving
natural communications to enable users to enjoy asense of reality without having to wear special devices
such as head mounted displays, 3D glasses, or headphones. In other words, the sophistication of environments surrounding users will make it possible toreproduce natural and realistic presence without burdening them. Such technologies could be applied in
the field of entertainment to create more realistic
experiences of sporting events or concerts. In business, these technologies should help teleconferencing
evolve from screen and audio sharing to space sharing. It may no longer be just a dream to have a remoteparticipant projected by a digital twin next to oneself
at a conference and whisper to that person or share
written communications. Even in the home, a television (TV) that plays back sound with volume andfrequency characteristics optimized only in areas
where there are elderly people could become a reality.
In other words, rooms and TVs present sounds properly controlled according to each listening area so one
would not have to increase the volume to suit the
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

elderly when watching TV with the whole family. It
may enable the elderly to enjoy sounds without hearing aids. It would also be possible to control thesound so that it would not leak in the direction ofchildren’s bedrooms so that one could watch TV
without the need of using headphones.
6. New approaches to transmit and present thefive senses + X directly and naturally
The evolution of sound and video as means of presenting information has been remarkable and has
made it possible to experience not only improvements in definition but also 3D effects. However,utilization of the other senses has not progressed.
Achieving natural user experience and presentationof various information requires utilization of visionand hearing as well as the other senses. Services thatcreate highly realistic experiences will have to appeal
not only to sight and hearing but also touch, smell,and even taste. In terms of information presentation,
methods that occupy the important sensory organs forsight and hearing are not always the natural means ofconveying information. This means that if information can be transmitted using methods other than42

Feature Articles

Vortices

Sense of blowing
Skin

Fig. 3. “ThermalBitDisplay” presents information to the skin.

visual or auditory, or a combination of these, it will bepossible to receive information naturally anytime andanywhere. We are currently working on presenting
information to the skin (Fig. 3). “ThermalBitDisplay” isan information-presentation device that uses temperature without disturbing the senses of sight and hearing and consists of a ring with an embedded thermoelectric element used to obtain information only
when pressed against the lips. Unlike push-type notifications, this device enables one to check information only when he/she wants to, and enables one tocheck information casually because one does notneed to use sight or hearing. We are also working on
the expression of new reality through stimulation
using vortex rings (air movements that can be generated using a so-called air cannon). By controlling thetime difference between stimuli, we are also researching technology to create pseudo-sensations as if
something is passing close to oneself. By creating
sensations other than that of something passing
through controls that include stimulation in certain
locations, for example presentations combining
sounds, we believe it will be possible to further
improve the realism of moving sound sources. Goingforward, we will continue researching and developNTT Technical Review

Vol. 18 No. 3 Mar. 2020

ing technologies to enable more natural experiences
or reception of information by conveying sensations
other than through the senses of sight and hearing.
7. Conclusion
This article described the research and developmentof natural communication technology to achieve freeinteraction and intersection between real and virtual
spaces including information exchange. The virtual
space is sometimes described as a mirrorworld [11],and we believe it is not just simply an electronic cyberworld but also a parallel world by intersecting withreal space. Ideally, we believe that as well as enablingusers to experience what they want to be good at,such as singing or dancing, in the parallel world, it isdesirable to feed this back to the real world. We will
make such intersection between virtual and realworld one of our themes and proceed with our
research and development of natural communication
technology.
References[1]

H. Nagata, H. Miyashita, H. Kakinuma, and M. Yamaguchi, “Real-time

43

Feature Articles

[2]

[3]
[4]
[5]
[6]
[7]

Extraction of Objects with Arbitrary Backgrounds,” NTT Technical
Review, Vol. 15, No. 12, 2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201712fa7.html
J. Nagao, H. Miyashita, T. Sano, K. Hasegawa, and T. Isaka, “Kirari!
for Arena: Real-time Remote Reproduction of 4-directional Views
with Depth Perception,” Journal of the Imaging Society of Japan, Vol.
58, No. 3, pp. 306–315, 2019 (in Japanese).
Website of Cabinet Office of the Government of Japan, “Society 5.0,”
https://www8.cao.go.jp/cstp/english/society5_0/index.html
I. J. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair,
A. Courville, and Y. Bengio, “Generative Adversarial Networks,”
Proc. of NIPS2014, pp. 2672–2680, Montreal, Canada, Dec. 2014.
Press release issued by NTT on Mar. 25, 2019 (in Japanese), https://www.ntt.co.jp/news2019/1903/190325b.html
T. Fukiage, T. Kawabe, and S. Nishida, “Hiding of Phase-based Stereo
Disparity for Ghost-free Viewing Without Glasses,” ACM Transactions on Graphics, Vol. 36, No. 4, pp. 147:1–17, July 2017.
M. Makiguchi and H. Takada, “360-degree Tabletop Glassless 3D
Screen System,” NTT Technical Review, Vol. 16, No. 12, 2018.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201812fa4.html[8] K. Tsutsumi and H. Takada, “Powerful Sound Effects at Audience
Seats by Wave Field Synthesis,” NTT Technical Review, Vol. 15, No.
12, 2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201712fa5.html[9] K. Tsutsumi, K. Imaizumi, A. Nakadaira, and Y. Haneda, “Analytical
Method to Convert Circular Harmonic Expansion Coefficients forSound Field Synthesis by Using Multipole Loudspeaker Array,” Proc.
of the 27th European Signal Processing Conference (EUSIPCO
2019), A Coruña, Spain, Sept. 2019.
[10] K. Imaizumi, K. Tsutsumi, A. Nakadaira, and Y. Haneda, “Analytical
Method of 2.5 D Exterior Sound Field Synthesis by Using Multipole
Loudspeaker Array,” Proc. of IEEE Workshop on Applications of
Signal Processing to Audio and Acoustics (WASPAA) 2019, New
York, USA, Oct. 2019.
[11] K. Kelly, “AR Will Spark the Next Big Tech Platform—Call It Mirrorworld,” Wired, Feb. 2019.
https://www.wired.com/story/mirrorworld-ar-next-big-tech-platform/

44

Feature Articles

Atsushi Sagata

Executive Research Engineer, Natural Communication Project, NTT Service Evolution
Laboratories.
He received a B.E. in electronic engineeringfrom the University of Tokyo in 1994 and joined
NTT the same year. He has been engaged in
R&D of video coding systems and in the development of the digital high-definition TV transmission system at NTT Communications. He is amember of the Institute of Electronics, Information and Communication Engineers (IEICE).

Takashi Sano

Senior Research Engineer, Natural Communication Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in electronicengineering from Tokyo University of Science in2002 and 2004. He joined NTT Cyber Space
Laboratories in 2004. He has been engaged in
R&D on software and hardware design of videocoding.

Kota Hidaka

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received an M.E. from Kyushu University,
Fukuoka, in 1998 and Ph.D. in media and governance from Keio University, Tokyo, in 2009. He
joined NTT in 1998. His research interests
include speech signal processing, image processing, and immersive telepresence. He was a seniorresearcher at the Council for Science, Technology and Innovation, Cabinet Office, Governmentof Japan, from 2015 to 2017.

Shinji Fukatsu

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received a Ph.D. in engineering from Osaka
University in 2002 and joined NTT the same
year. He has been engaged in R&D of humaninterfaces and video streaming services. He has
also been engaged in planning and developmentof video streaming services at NTT Plala and inpromoting standardization and ICT international
development at the Ministry of Internal Affairs
and Communications. He is currently engaged in
R&D of natural communication technology.

Takafumi Mukouchi

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received an M.E. from Waseda University,
Tokyo, in 1995 and joined NTT the same year.
He has been engaged in the research of machinelearning and development of video communication systems. He is currently engaged in the
research of natural communication technology.
He is a member of IEICE.

Hidenobu Nagata

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received an M.E. in systems and information engineering from the Faculty of Engineering, Hokkaido University in 2001. He joined
NTT in 2001 and studied video handling technologies including video searching, video indexing, automatic summarization and their interfaces. From 2008 to 2014, he worked at NTT Electronics and developed professional transcoders
and embedded audio IP for consumer devices. He
transferred to NTT in 2014 and is currently
researching ultra-realistic telecommunication
technology.

Takashi Satou

Senior Research Engineer, Supervisor, Natural
Communication Project, NTT Service Evolution
Laboratories.
He received a Ph.D. in information engineeringfrom the University of Tokyo in 1996. He joined
NTT the same year and studied human computerinteraction and video handling technologies. He
managed R&D of image recognition technologies and innovation process in NTT DOCOMO
from 2014 to 2019. He is currently managing
R&D of natural communication technologies in
NTT Service Evolution Laboratories.

NTT Technical Review

Vol. 18 No. 3 Mar. 202045

Feature Articles: ICT R&D Initiatives for a Smart World
that Harmonizes People and Environments

Ambient Assistance Technology
Supporting Human Behavior by
Understanding the Relationshipbetween People and Environments
Hitoshi Seshimo, Yoshiki Nishikawa, Takao Kurahashi,
Yukio Koike, Osamu Matsuda, Shin-ichiro Eitoku,
Tomoki Watanabe, Tae Sato, Takashi Isezaki,
and Mana Sasagawa
Abstract

Many of the actions we take are not only based on our own abilities, thoughts, and emotions, but arealso broadly influenced by the real-world environment that surrounds us. Focusing on the environmentsthat surround people engaged in activities, we aim to establish ambient assistance technology to help
people choose the most optimal behavior by natural interaction with the environment based on models
that combine environments with human perceptive and physical states.
Keywords: human/environment sensing and modeling, behavior simulation, action support

1. Introduction
Our “Point of Atmosphere” world, where naturalinteractions that harmonize people and environments
are possible, is composed of information and communication technologies (ICTs) that help peoplechoose the best behavior. These ICTs can determine
the situations people are in, objects surrounding
people, and their relationships with others as well as
their behaviors, intent, and feelings in a wide range ofdaily activities. This suggests that ICT will go beyond
online services and evolve to provide value by directly influencing various human behaviors in the realworld (Fig. 1).
Based on the trends in this ICT evolution, we areinvestigating ambient assistance technology, whichmodels real-world environments (spaces, objects, andother people) that surround people engaged in various
activities. The constructed models are combined withNTT Technical Review

Vol. 18 No. 3 Mar. 2020

the models of perceptional, psychological, physiological, and physical states of people. Then, thechoice of the best action is naturally supported
through the devices that are human-environment
interfaces (Fig. 2).
2. A new world born of ambientassistance technology
What new conveniences will come about in ourlives through the development and use of ambientassistance technology? In a city, for example, the possibility of colliding with a dangerously driven car
when that car is in the pedestrian’s blind spot can be
determined using models of the person and thosearound him or her as well as environmental models
for roads and traffic flows to instantly derive a simulation to enable the person to take advanced precautions, such as evacuating to a nearby building,46

Feature Articles

PersonEnvironment

Environment(Office)(Home)

Environment
PersonWeb service
Web service
Web service

Environment
Environment(Town)Web service

Currently: An era centered on online services

From now: An era centered on real world services• Using smartphones or personal computers toaccess services (environments) on the Internet
and enjoy the benefits• Enjoying the benefits of various behaviors of people
open to the real world through natural approaches
from anywhere

Fig. 1. Evolution of ICT.

Naturalinfluence

Body

Geospace

MemorySensesHuman states- Where a person is- What a person is doing- What a person can see- What a person can hear
- Physiological states-…

Choosing optimal
behavior
×Interaction

Other party

Thing

Surrounding environments- Geospatial- Meteorological- Positions of things- Existence of other parties-…

Fig. 2. A world enabled by ambient assistance technology.

through digital signage around the person or wearable devices. For those who want to lose weight, a
map app will guide them on a route with a slight
detour to appropriately increase their activities without affecting the arrival time at their destinations, andan artificial intelligence agent could present easy-toaccept and optimized exercises to a person with theright timing to spark motivation and help exercising
become a habit. In the office, these technologies
enable environmental measurements to, e.g., increase
productivity and encourage creative collaboration byadjusting temperature for comfort, adjusting lighting(brightness, color) to encourage concentration or
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

relaxation, and creating sound fields to raise confidentiality or conversely to encourage informationexchange, based on the modes of behavior in whichpeople work (at their desks, in meetings, brainstorming, presentations, etc.) and surrounding situations(room layouts, desk alignments, gatherings of people,
office equipment arrangements, etc.). This will naturally guide people towards the best actions to enable
them to casually notice things that they would notnormally notice, enable them to change their behaviors to achieve their goals, naturally change the way
they work with influence from the environment, andincrease their productivity and promote creative47

Feature Articles

Sensing people andenvironments
to collect data about one’s
surrounding environment

Modeling people andenvironments
to simulate activities

Feedback for behavior
support to encourage
optimized behaviors

Ambient sensing technology

Real-world modeling technology

Ambient feedback technology

Physical laws and traffic rules, etc.
Direct feedback to
peopleInformation onhuman states andsurrounding
environments

Indirect feedback to
people through theenvironment
Space
Other
partiesThing Robot
One’s self

Environment

Sphere of one’s activity

Fig. 3. Elements of ambient assistance technology.

collaboration.
3. Towards ambient assistance technology
To make the aforementioned world a reality, we aredeveloping ambient assistance technology to support
optimized human behaviors. To do this, the following
technical elements are required (Fig. 3). The first element is ambient sensing technology. Human behavior
is based on various abilities, such as physical andpsychological, that mutually influence each other.
Additionally, environments in which people are partaking in various activities and their relationships
with others are also considered to affect the demonstration of their abilities. Ambient sensing technology
enables the generation of models for people and theirsurrounding environments by continually collecting
and digitizing status information of the activity areas
centered on active people without interfering with
their original activities. The second element is realworld modeling technology. Ambient sensing technology collects information on the states of active
people and their surrounding environments to reveal
the mechanisms (models) of the changing states of
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

people’s abilities or physical spaces. Real-worldmodeling technology integrates such digital models
of people and the environment to enable reproduction(simulation) of real-world situations in the past, present, and future. The third element is ambient feedback technology. Real-world modeling technology
can simulate human behavior and psychologicalabilities in certain environments and situations.
Ambient feedback technology supports user behaviors by extracting optimal behaviors in a situationsuch as specific training methods and influencing theuser through wearable devices and devices embeddedin the surrounding environment.
4. Ambient sensing technology
Ambient sensing technology focuses on humanbehaviors and the states of their surrounding environments in the real world and digitizes them immediately. Specifically, while defining targets of humanand environmental sensing, we are studying the following technologies: personal space life log collection technology that continuously collects and digitizes state information of the activity area (personal48

Feature Articles

space) in which the person is without causing themstress; personal space life modeling (twin modeling)technology to generate personal modeling (behavior,
body movement, perception, emotions, physiology
and ecology, etc.) and surrounding environment modeling (physical space, environmental state, surrounding object recognition, communications, etc.). For
example, for human sensing and modeling, we havebeen working on state estimation (central fatigue
estimation [1, 2] and state transition estimation ofmuscle activity patterns [3]) based on the collection
and analysis of biometric data including heart rate
and myoelectric data collected from wearable devices
worn on the body in the form of shirts, socks, hats,
etc. For sensing and modeling the environment, wehave been working on technologies that estimate roadconditions a user is currently walking on (inclinations, steps, stairs, etc.) obtained from body movements through the smartphone the user is carrying [4]and technology that maps sensing information frommany people to physical space and detects changes inenvironmental conditions over a wide area [5]. Starting with devices to simultaneously record personaland environmental information acquired to date, weplan to study new sensing and modeling approaches
combining environments and people that take into
account the abilities of a person appearing in an environment, changes in the states of the environments
appearing with a person’s behavior, etc.; thus,
encouraging further research and development of
ambient sensing technology.
5. Real-world modeling technology
Real-world modeling technology accumulates personal and environmental twin models obtained fromambient sensing technology and lays over high-precision spatial information and environmental monitoring, which is a bird’s-eye-view real-world model
contrasting personal space, and incorporates physicallaws, traffic rules, etc to achieve an autonomous realworld model that is constantly updated. In the realworld model, it can be expected that higher-ordersituations (contexts) will become clear from thereproduced relationships between people andbetween people and the environment. For example, a“cognitive map” that shows how a person perceives
his or her place in geospatial space is dependent on
his/her spatial cognitive ability, sense of direction,and spatial knowledge, which vary from person toperson. It is also said that a place can be felt to berelatively closer than the actual geographical distance
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

depending on the familiarity with it and attractiveness[6, 7]. Thus, reproducing real-world maps that lookdifferent and are distorted for each person from their
spatial cognitive models and high-precision spatial
information will reveal what a person who gets lost
easily is actually looking at and in what kind of situations and what intentions caused them to lose theirway, which can then be used to naturally help the
person avoid such situations. Also, it could be possible to reproduce human memory models for changes
in time by reproducing how information is transferred
between people and environments in the real world
by simulating past-present-future time lines fromaccumulated real-world models. With real-worldmodeling technology, therefore, we plan to investigate the digitizing of things that cannot be directly
measured with individual sensors and models.
6. Ambient feedback technology
Ambient feedback technology supports people toengage in optimal behaviors calculated using realworld modeling technology. This technology will
require methods to simulate human behavior andselect optimal behaviors based on those simulation
results. Therefore, we are considering using realworld models not only of oneself but also of otherpeople. We aim to calculate optimal behaviors bycomparing other people’s behaviors calculated fromthe models of others. This other person behavior issimulated by applying the same situation and purposeof behavior as oneself to the model of the other person. For example, by comparing the behaviors ofsports professionals with one’s own behavior it might
be possible to calculate more specific training methods. People will require support in performing behaviors after calculating optimal behaviors. Smartphone
event notifications are often used for this purpose. In
the future, we will consider more natural and casual
methods so that anyone can benefit regardless of ICT
literacy. For example, if the digital signage now
widely in use advances further, anything in a city
could become a signage device. If so, it might be possible to provide timely messages that are easy forpeople to perceive not only through the visual andauditory senses but also through any or all five senses.
In addition, if wearable devices evolve, it might bepossible for people to engage in optimal behaviors byjust wearing such a device.
We conducted research on comparing humanbehavior by extracting the characteristics of professional and amateur movements [3]. In terms of49

Feature Articles

wearable devices, we conducted research on inducingchanges in muscle activity and walking without any
user intention by applying contact stimulation to thesole of the foot from the insoles of shoes that change
in hardness and shape [8]. To encourage people toengage in optimal behaviors, we have been researching message-generation methods that are easy forpeople to accept, based on behavioral change models[9]. We will keep expanding these techniques toenable the implementation of ambient feedback technology.
7. Conclusion
Although efforts to develop ambient assistancetechnology have only just begun, we are aiming for aworld in which anybody can enjoy its benefits regardless of their ICT literacy. We will work quickly andsteadily towards achieving the challenging goals of
deeply understanding people and environments in thereal world and modeling them to extract optimized
behaviors of people from simulations and support
people in engaging in those behaviors naturally.
References[1]

T. Kondo, Y. Yamato, M. Nakayama, A. Chiba, K. Sakaguchi, T.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

[2]

[3]

[4]

[5]
[6]
[7]
[8]

[9]

Nishiguchi, T. Masuda, and T. Yoshida, “Natural Sensing with ‘hitoe’
Functional Material and Initiatives towards Its Applications,” NTT
Technical Review, Vol. 15, No. 9, 2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201709fa3.html
K. Eguchi, R. Aoki, S. Shimauchi, A. Chiba, and N. Asanoma, “Biosignal Processing Methods Targeting Healthcare Support Services,”
NTT Technical Review, Vol. 16, No. 8, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201808fa5.html
T. Isezaki, R. Aoki, T. Indo, S. Deshpande, and L. Tamir, “Transition
Characteristics Analysis of Muscle Activity Patterns for Firefighters,”
Proc. of the 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 23–27, Berlin,
Germany, July 2019.
Y. Kurauchi, N. Abe, H. Konishi, and H. Sehimo, “Barrier Detection
Using Sensor Data from Multiple Modes of Transportation with Data
Augmentation,” Proc. of the IEEE 43rd Annual Computer Softwareand Applications Conference (COMPSAC 2019), pp. 15–19, Milwaukee, USA, July 2019.
Press release issued by NTT on Nov. 22, 2018 (in Japanese).
https://www.ntt.co.jp/news2018/1811/181122b.html
Y. Wakabayashi, “The Evolution of Map: Geospatial Information and
Future of Humans,” Sogensha Inc., 2018 (in Japanese).
Y. Wakabayashi, “Spatial Analysis of Cognitive Maps,” Chijin Shobo,
1999 (in Japanese).
M. Sasagawa, A. Niijima, K. Eguchi, R. Aoki, T. Isezaki, T. Kimura,and T. Watanabe, “Lower Limb Muscle Activity Control by using
Jamming Footwear,” EMBC 2019, pp. 3302–3305, Berlin, Germany,
July 2019.
T. Sato, R. Aoki, M. Koyasu, I. Shinozaki, N. Oshima, N. Mukawa, T.
Watanabe, and T. Indo, “Koudouhennyou no Tameno Ninchiteki
Fukyouwa ni okeru Message Teijihouhou (A Message Presentation
Method in ‘Cognitive Dissonance’ for Behavior Change),” DICOMO2019, pp. 670–675, Fukushima, Japan, 2019 (in Japanese).

50

Feature Articles

Hitoshi Seshimo

Senior Research Engineer, Supervisor, Networked Robot & Gadget Project, NTT Service
Evolution Laboratories.
He received a B.E. and M.E. in mechanicalengineering from Waseda University, Tokyo, in
1995 and 1997. He joined NTT in 1997. His
research interests include computer aided
instruction, web-based learning, content distribution and navigation systems, and geographicalinformation services.

Yoshiki Nishikawa

Senior Research Engineer, Supervisor, Networked Robot & Gadget Project, NTT Service
Evolution Laboratories.
He received an M.E. from Waseda University,
Tokyo, in 1998. He joined NTT in 1998. His
research interests include human computer interaction and robotics.

Takao Kurahashi

Shin-ichiro Eitoku

Senior Research Engineer, Networked Robot& Gadget Project, NTT Service Evolution Laboratories.
He joined NTT in 2006 and is currently
engaged in researching information systems andmultimedia systems for communications.

Tomoki Watanabe

Senior Research Engineer, Networked Robot& Gadget Project, NTT Service Evolution Laboratories.
He received a B.E. from Yokohama National
University, Kanagawa, in 1992, and a Ph.D. from
Kanagawa Institute of Technology in 2015. He
joined NTT Human Interface Laboratories in1992. His research interests include human computer interaction and network technology for IoT
services.

Tae Sato

Senior Research Engineer, Networked Robot& Gadget Project, NTT Service Evolution Laboratories.
He received an M.E. from Hosei University,
Tokyo, in 2002. He joined NTT in 2002. His
research interests include human computer interaction and robotics.

Researcher, Networked Robot & Gadget Project, NTT Service Evolution Laboratories.
She received an M.E. in engineering from
Tokyo Institute of Technology in 2010. She
joined NTT in 2010. Her research interests
include psychology and human computer interactions.

Yukio Koike

Takashi Isezaki

Senior Research Engineer, Networked Robot& Gadget Project, NTT Service Evolution Laboratories.
He received an M.S. from Nagoya University,
Aichi, in 2004. He joined NTT in 2004. His
research interests include information systems
and network technology for Internet of Things(IoT) services.

Researcher, Networked Robot & Gadget Project, NTT Service Evolution Laboratories.
He received a B.E., M.E., and Ph.D. in engineering from University of Tsukuba in 2011,2013, and 2019. He joined NTT Service Evolution Laboratories in 2013. His research interests
include biological signal processing, ambientsensing, and cognitive science.

Osamu Matsuda

Mana Sasagawa

Senior Research Engineer, Networked Robot& Gadget Project, NTT Service Evolution Laboratories.
He received a B.S. from Aoyama Gakuin University, Tokyo, in 1994. He joined NTT in 1994.
His research interests include sensing and modeling the surrounding space and environment.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Researcher, Networked Robot & Gadget Project, NTT Service Evolution Laboratories.
She received an M.E. in engineering from
Ochanomizu University, Tokyo, in 2017. She
joined NTT in 2017. Her research interests
include human computer interactions and ubiquitous computing.

51

Feature Articles: ICT R&D Initiatives for a Smart World
that Harmonizes People and Environments

User Experience Method for Deeper
Understanding People’s Values to
Design and Change Their Behavior
Yumiko Matsuura, Hiroyuki Toda, Momoko Nakatani,
Takeshi Kurashima, and Harumi Saito
Abstract

Advances in technology has expanded the scope of people’s lives even though the behavior of people
living comfortable lives has become complicated along with various personal values systems. This
article introduces methods with which people can naturally achieve a state of well-being faster byenabling them to adopt technology to change their behavior for the better. There are two methods, one
for analyzing people’s daily behaviors or Internet usage for understanding people and the other for providing technology to promote awareness and a sense of achievement by clarifying mechanisms of change
regarding people’s minds and behaviors.
Keywords: behavior design, understanding human behavior, well-being

1. Behavior design to achieve a stateof well-being
The recent spread of the Internet of things (IoT), in
which all things are connected through the Internet,
has brought closer a world in which it is possible todetermine what is happening now, what will happen
in the future, and how objects, e.g., devices, should
move towards obtaining desirable results, based on
multi-dimensional, large-scale data obtained from
IoT that far exceeds the processing capabilities ofhumans. If ambiguous and complicated humanbehaviors can be understood using artificial intelligence (AI), it will likely become possible to plan not
only how objects but also people should move. AI canstudy what is positive behavior from their behaviors
to provide feedback for them to change their behavior
for the better. If this cycle can be sustained, people
can achieve a state of well-being [1] (Fig. 1).
Therefore, with findings from our previous studies
on (for example) service-design-process research andanalysis and modeling of spatio-temporal behaviors,
we aim to develop interaction technology that will
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

influence human cognition to enable people toachieve a state of well-being so they can continue ina more positive direction. This technology will beachieved by scaling various ambiguous and diverse
values such as prioritizing objects—not only economic values, understanding ideals that people pursue, observing what behaviors people engage in toreach their ideals and modeling them, and enabling
the prediction and measurement of success levels
from those models. If data from various people can beacquired through IoT devices, through those observed
data and interviews regarding people’s ideals anddaily activities, the values they have, and how thoselevels are expressed can be handled in models of user
values. It will also be possible to deepen understanding of the behavioral models that explain why a person takes a certain action and what principle led tothat action from the actual behavioral data. Oncemodeled, the current situation can be fit to the model,
what will happen from the fitted model can be predicted, and how results will change with what actions
can be understood so that results of action towards an
ideal can be predicted. Then, by understanding how
52

Feature Articles

Digitalization

Real world

Modeling

EffectMultidimensional,
large scale

Automatic controlof objects

Humans have to process large
quantities above theircapabilities.

Cities, environments,
objects
PeopleHumans have to understanddifferent types of actions
beyond their knowledge.
Ambiguity,
complexity

To support human choiceFig. 1. Changing behavior to achieve a state of well-being.

people accept information and feel about conversation and communication with other people, etc. andby providing a means to give them awareness, behaviors can be designed that will lead people to predicted
results that approach overall and individual ideals sothat environments to assist people in reaching a stateof well-being can be created (Fig. 2).
These environments could work in concert daily to
encourage behaviors under various conditions where
there is a need for individual or group optimized
behavior, such as finding the shortest time when moving, avoiding congestion, or wanting to visit various
places. This could lead to a less crowded society as awhole. Also, through dialogue between people andstaff at helpdesks, shops, or neighbors, this could leadpeople in a direction that satisfies not only the individual but also stakeholders. To induce positive
behaviors without one’s (people’s) awareness, similar
to if one was moving by one’s own will, we aim tosupport people’s natural behavior to smoothly andcomfortably improve themselves by providing necessary information if they become confused and supporting those with diminished physical capabilities.
When conducting daily activities will be possible inboth the real and virtual worlds, it might be possible
to have personalities in both. Even if a person uses
different personalities in both, has several living
spaces including virtual space, and their values
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

become more diversified, we will be able to promoteresearch and development for assisting people
towards positive behavior more naturally. We hope
this interaction technology will improve people’s
lives.
2. Technology to achieve behavioral design
2.1 U nderstanding human behaviors for achieving a state of well-being
To lead a person to a state of well-being, it is important to understand the behaviors (e.g. moving from A
to B, staying at A, eating, etc.) and states (e.g. physical and mental conditions, etc.) of that person and thesurrounding environment. As mentioned above, datacollection has been rapidly progressing due to thewide spread of IoT. Thus, we are pursuing researchand development based on the framework shown in
Fig. 3, aiming at a world that uses data to bring people into a state of well-being. The most important
point is not only to understand the behavior recordedin observation data but also construct a behavior
model based on those data. Since this understanding
and modeling make it possible to understand the
behavior of people in situations in which there are noobservational data and predict future behaviors, we
believe it will be easier to decide on strategies tochange the future for the better.
53

Feature Articles

Modeling the diverse values of users

Behavioral
observation(psychology)

Modeling behavioral decisions of diverse values
Various users

Machinelearning(statistics)

User-value modelingmatch judgment
model decisions

Human value
modelsBehavior judgment
model

Behavior design assistance to reach theirstates of well-being

Subsequent behavior planning
through judgment model

Adaptive extraction andclarification of valuesYour current self

State of well-being

Supplement

Behavior support toaccelerate the achievementof state of well-being
Raising capabilities matching values

ExpandFig. 2. Behavior-change support to reach a state of well-being.

Knowledge about
people’s behavior
from the social
science perspective

Data from observing
people’s behavior

Best directions(social requests,
personal preferences,etc.)

Knowledge discovery

Modeling

Policy decisions

Interaction

Uncover knowledge
about people’s behaviors
or peripheral
environments byanalyzing measured data.

Enable prediction ofsituations for which there
are no data throughmodeling that combines
knowledge obtained fromdata and from social
science.

Plan measures to guidepeople based on best
directions and modelbased predictions.

Take measures to guidepeople to state of wellbeing in a way that isbest suited to theirsituation.

Fig. 3. A framework that uses data for people to achieve a state of well-being.

To achieve this, people flow estimation (PFE) [2]makes it possible to understand the people flow not
explicitly included in data by using the characteristics
of human behavior obtained from data analysis asknowledge.
As shown in Fig. 4, from population information ofeach area at each time step, PFE estimates the people
flow between cells over time. The point with thismethod is that it estimates the people flow between
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

areas from only the population information of eacharea at each time step, which does not explicitly represent crowd movement. When the number of areas
where people are moving is N, this method solves the
problem of estimating people flow between areas(number of unknown values (transition in population
between each area pair) is N2) from population information for two time steps (number of known values(population of each area) is N × 2). If this problem is54

Feature Articles

Input: Population Nti in Grid i at time t
7:00 am

8:00 am

9:00 am

90

60

40

60

70

60

40

10

80

10

20

40

20

20

40

10

20

20

30

40

80

40

40

60

60

30

20

Output: No. of people Mtji moving from Grid i to Grid j within time t  Time t+1
8:00 am – 9:00 am

7:00 am – 8:00 am
10

20

10

20
10

10

10

Fig. 4. Estimating people flow from spatio-temporal population data.

considered, there will be too many solutions that areconsistent with the observed population data. Since
there will be multiple movement patterns that can
explain the change in population for each observed
area of the input data, it will be difficult to accurately
estimate the correct people flow.
In response to this problem, we first clarified thecharacteristics of human behavior in cities by analyzing real data. We then introduced the characteristics
of human behavior obtained from data analysis as aconstraint to the problem of estimating people flow
between areas to narrow down the solutions.
Specifically, we used the following findings, whichwere obtained based on people-moving data in cities.
•	Human movement is affected by the distance
between areas (Saitama → Tokyo > Saitama →Yokohama)•	People’s departure probabilities vary from area
to area (weekday mornings: residential district>business district)•	The ease with which people gather depends on
the area (weekday mornings: business district>residential district).
Based on the above findings, instead of directly
estimating the population between areas (number of
unknown values is about N2), we estimate scores thatreflect the characteristics of each area (e.g. score of
how easily people gather in each area, score of how
likely people depart from each area; number of
unknown values is about N × 2) that affect the movement of people (Fig. 5). PFE uses the following formula to understand the flow of people.
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

(Ease of moving from area i to area j) ≈ (probability of departure from i) × (ease of people gathering atj) × (ease of movement based on the distance between
i and j).
This model makes it possible to select a solutionthat seems to move people. It is also possible to drastically reduce the unknown values that need to be
calculated, resulting in much lighter computation
cost.
The above is an example of incorporating theknowledge obtained from data analysis into a model
to more accurately estimate human behavior without
explicit observed data of the behavior. In addition to
the knowledge gained from data analysis, we will
also try to model human behavior using social science knowledge. We believe that this modeling will
enable us to understand various human behaviors toeffectively change them for the better.
2.2 R
 esearch of understanding human values andinteraction for achieving a state of well-being
Clarifying the mechanism of well-being is obviously not something that can be done overnight. A
state of well-being is very different for each person
and changes depending on the surrounding environment and circumstances. Moreover, it is well known
that people’s thoughts and judgments are often irrational. To achieve our goal of clarifying the mechanism of well-being, we are conducting research toboth deeply understand the diverse values of people
and develop support methods (interaction technology)
by using knowledge from a wide range of fields such55

Feature Articles

Incorporating characteristics of the moving behavior of people makes it possible toestimate the people flow between areas more accurately.
Examples of possible people-flow patterns
Observed
population changes
Tokyo Station

Wrong
Unnatural for so many
to be moving far away

Wrong
People normally gather at Tokyo
Station, but there is no movementfrom Shinagawa to Tokyo.
Shinagawa Station
Increase
Decrease
No change

Correct

Fig. 5. Observed population changes and examples of possible people-flow patterns.

as psychology, engineering, and design (research tounderstand values).
To understand people’s values related to wellbeing, our approach is to clearly define targets andstudy people’s values both quantitatively and qualitatively. For example, when we previously focused onparents who are raising small children, we conductedboth web surveys and interviews to understand theirvalues and child-rearing behaviors [3]. The results
indicated that parents have both short-term and longterm goals. There are six long-term goals regarding
their children’s futures. Figure 6 illustrates the general results. Some parents wanted to entertain or calm
their children (“To make the kids feel calm andhappy” in the diagram). These factors can be
explained as short-term goals. The same parents may
also whish their children “To be a person who can dowhat s/he wants (goal 1)” or “to be a person who is
independent-minded in his/her life or choices (goal2)”, which can be explained as long-term goals. Having both long and short-term goals, items that are
given priority differ depending on the state of the parent’s mind, environments, and the state of the child,
which may lead to disparate child-rearing behavior.
As shown in Fig. 6, parents’ long-term goals fortheir children’s future can be categorized into six
types with the degree of emphasis depending on theparent. There are also multiple types of “good life
goals” positioned above the long-term goals. The
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

various factors make it difficult to understand thereason of behavior. However, our approach of using
both qualitative methods, such as interviews, andquantitative methods, such as large-scale questionnaires, enable us to deeply understand human values.
In the case of understanding parents, after modelingvalues, we developed scales to enable easy measurement of value types, which would lead to developing
support services that match parents’ values.
The reason we focus on interaction research is
because people are social creatures, and interactions
with others have a significant impact on the state ofwell-being. Therefore, we are focusing on the effects
of interactions among diverse people and developing
methods for achieving a state of well-being. Since it
is important to understand the interactions amongpeople and what changes the motives and thoughts ofpeople, we are conducting research in the real world
on limited targets and situations to observe them as
naturally as possible. We are mostly designing anddeveloping tools/programs to support dialogue inliving labs. Living labs is an approach that involve
users as co-creators in the service-design process. It
is attracting attention not only for service design but
also as a methodology useful for community development including local revitalization. It also contributes
to a state of well-being. Incorporating living lab
activities in the community enables people to livewith better awareness. Hence, we are implementing
56

Feature Articles

Goals for a good life

• A more enjoyable life
• A life of achieving greater things• A life of growing as a person and achieving a state of well-being• An easy life and avoiding the worst

Long-term goals
Goal 1
To be a person
who can dowhat s/hewants to do

Subgoals(omitted)

Short-term goals
Goal 2
Goal 3To be a person To be a person
who iswho has a
independentwarm feelings
minded intoward others
his/her life or
and can act
choicesfor others
Subgoals(omitted)

Subgoals(omitted)

Goal 4To be a person
who is a
respectable
member ofsociety
Subgoals(omitted)

Goal 5
Goal 6
To be a person To be a person
who
who can trust
understands
parentsand supports
others aroundhim/her
Subgoals(omitted)

Subgoals(omitted)• To entertain the kids now• To make the kids feelcalm and happy now• To promote the kids’
development now• To meet their own needs

Child-rearing behaviors

Fig. 6. Values of parents that affect their child-rearing behavior.

living labs in multiple locations, such as in the Japanese cities of Omuta, Sendai, and Yokohama, while
engaging in dialogue with partners with differingvalues such as residents, companies, local governments, and universities. For example, at Tama-Plazain Yokohama, we are studying how to support andaccelerate the efforts of the local residents, companies, and government by developing frameworks andtools to communicate with one another to solve various problems the town has. We are also conducting
research in Omuta, a city that places importance on a
person-centered view of people that integrates theidea of “living true to oneself” while remaining connected with surrounding people, in collaboration withthe local government, organizations, and companies
to understand what is required to improve the town sothat people can live in it with this view. Through thisfield research, we are painstakingly clarifying important viewpoints and developing frameworks and tools
to support people in interacting with one another toachieve states of well-being.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

3. Future developments
People’s values and behaviors are diverse andsometimes irrational. Understanding, modeling, andtransforming them into a better state is certainly avery difficult challenge. We will continue to engage
in research and development with the aim of improving the well-being of people and solving social issues
by using knowledge from a wide range of fields suchas psychology, data analysis, and design.
References
[1]

[2]

[3]

J. Watanabe, Y. Ooishi, S. Kumano, M. Perusquía-Hernández, T. G.
Sato, A. Murata, and R. Mugitani, “Measuring, Understanding, andCultivating Wellbeing in the Age of Technology,” NTT Technical
Review, Vol. 16, No. 11, 2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa6.html
Y. Akagi, T. Nishimura, T. Kurashima, and H. Toda, “A Fast andAccurate Method for Estimating People Flow from Spatiotemporal
Population Data,” Proc. of IJCAI-ECAI 2018, Stockholm, Sweden,
July 2018.
A. Nakane, H. Watanabe, M. Watanabe, Y. Katagiri, and T. Kobayashi,“The Structure and Characteristics of Japanese Mothers’ Child-rearing Goals: In-depth Interviews Revealed Six Long-term Goals,”
IEICE Trans. Fundamentals (Japanese Edition), Vol. J102, No. 2, pp.
48–58, 2019.

57

Feature Articles

Yumiko Matsuura

Senior Research Engineer, Supervisor, Universal UX Design Project, NTT Service Evolution
Laboratories.
She received a B.S. and M.S. in computer science from Keio University, Kanagawa, in 1991
and 1993. She joined NTT Human Interface
Laboratories as an engineer in 1993 and engaged
in the development of multimedia systems and acontent delivery platform. She was in the R&D
Strategy Department twice, most recently she
started collaboration with leading companies inindustries such as automotive and manufacturing. As a result of organizational changes in July
2017, she is now at NTT Service Evolution Laboratories and involved in research management on
human understanding and interface design.

Hiroyuki Toda

Senior Research Engineer, Supervisor, NTT
Service Evolution Laboratories.
He received a B.E, and M.E. in materials science from Nagoya University in 1997 and 1999,
and a Ph.D. in computer science from Universityof Tsukuba in 2007. He joined NTT in 1999. His
current research interests include information
retrieval and data mining. He is a member of theInformation Processing Society of Japan (IPSJ),
the Institute of Electronics, Information andCommunication Engineers (IEICE), the Japanese Society for Artificial Intelligence, the Database Society of Japan, and the Association for
Computing Machinery (ACM).

Takeshi Kurashima

Senior Research Engineer, Universal UX
Design Project, NTT Service Evolution Laboratories.
He received a B.E. from Doshisha University,
Kyoto, in 2004, and an M.S. and Ph.D. in informatics from Kyoto University in 2006 and 2014.
He joined NTT in 2006 and has been studying
human-behavior data mining and analysis. From2016 to 2017, he was a visiting scholar at Stanford University, USA. He is a member of ACM
Special Interest Group on Knowledge Discovery
and Data Mining, IPSJ, and IEICE.

Harumi Saito

Senior Researcher, Universal UX Design Project, NTT Service Evolution Laboratories.
She received a B.A. and M.A. in psychology
from the University of Tokyo in 2004 and 2006,
and a Ph.D. in informatics from Yokohama
National University in 2019. She joined NTT in2006 and worked on applied research concerninguser-friendly graphical interface design, especially color design for people with color vision
deficiencies. From 2016 to 2019, she worked asa UX researcher/consultant at NTT TechnoCross
Corporation. She is a member of IEICE, theJapanese Psychological Association, the Color
Science Association of Japan, and the Human
Interface Society.

Momoko Nakatani

Distinguished Researcher, Universal UX
Design Project, NTT Service Evolution Laboratories.
She received an M.E. and Ph.D. in engineeringfrom Waseda University, Tokyo, in 2003 and2013. She joined NTT Service Evolution Laboratories in 2003 and researched human computerinteraction, human-centered design, manual
design, and human modeling. In 2014, she started a user experience (UX) consulting business at
NTT-IT (now NTT TechnoCross Corporation).
Since returning to NTT Service Evolution Laboratories in 2017, she has been studying the UX
design process, living labs, and group collaboration.

NTT Technical Review

Vol. 18 No. 3 Mar. 202058

Feature Articles: ICT R&D Initiatives for a Smart World
that Harmonizes People and Environments

Sports Social View—Method of
Inclusive Sports Appreciation
Extracting and Sharing the Essenceof Sports through the Sense of Touch
Akiko Hayashi, Junji Watanabe, and Kentaro Shimizu
Abstract

NTT Service Evolution Laboratories is collaborating with the Tokyo Institute of Technology to enablepeople with various physicalities, including those with disabilities, to enjoy sports together. This article
introduces a new method of enjoying sports that not only uses the audiovisual senses but also the senseof touch.
Keywords: tactile information, inclusive design, translation of sports

1. For inclusive sports viewing
What can be done so that people with various
physicalities, including people with disabilities, canenjoy watching sports together? Understanding
diversity has become increasingly important as
reflected in the vision “diversity and harmony” of theinternational sports event to be held in Tokyo in 2020.
NTT Service Evolution Laboratories is collaboratingwith Associate Professor Asa Ito of the Tokyo Institute of Technology on research into sports viewing
for the visually impaired. We started with the aim tosupport people with disabilities to enjoy sports, but as
the joint research progressed, rather than support, wearrived at the awareness of accepting differences andco-creating new experiences.
2. The limitations of describing things
with words
The most common way for the visually impaired toenjoy sporting events is through audio commentary.
There are people who enjoy audio commentary ofbaseball or soccer. However, some visually impaired
people who once were able to see before have said,
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

“We were able to enjoy exciting scenes as we saw
them,” but after their sight deteriorated they described
their sports enjoyment as only a contextual overview
of the flow of matches through audio commentary. In
other words, they became unable to enjoy the raw
skills of the players. There are also limitations toaudio commentary for some competitions. Althoughcompetitions, in which offense and defense taketurns, such as baseball, can be understood throughaudio commentary, it is difficult to convey the finedetails, such as the force of movements and rhythm,
in competitions such as tennis with its dizzying rallies
or judo in which offense and defense are deployed at
the same time.
A sense of unity is also an important factor affecting satisfaction when watching sports [1]. This means
getting excited at the same time as a spectator nearby.
However, it is difficult for the visually impaired tofeel a sense of unity with the surrounding spectators.
Therefore, current audio commentary is problematicin that it cannot express movements in sports very
well, and the visually impaired are left out of the surrounding excitement.

59

Feature Articles

Fig. 1. Translation of tennis match.

3. Physical translation of sports
To enable sharing of sport-viewing experienceswith the visually impaired, we developed a method ofviewing sports that physically re-expresses (translates) what is happening instead of using language.
With this method, we attempted to express the force
and rhythm of movements during a tennis match andjudo match, which are difficult to verbalize.
3.1 Tennis match
The translation of the tennis match involved a translator (a sighted person) and visually impaired person(with total late-onset blindness) sitting face-to-facewith the translator tapping a circular board laid acrossboth their knees (Fig. 1) to reflect the position andstrength of the ball as it was hit while the translator
watched the tennis match. During a rally, the board
was struck rhythmically on the visually impaired person’s left and right, and he moved his head left andright as if he were following the movement of the ballto watch the rally. When a ball was hit and had a longflight, the visually impaired person could be heard
uttering “this is a big one,” indicating that he was
enjoying the spatial nature of the rally. The translator
also conveyed not only the physical action of hitting
serves but also the rhythm of the players before serves
by tapping the board. Although such rhythm does notappear as actual movement, the translation of tension
and rhythm helps the visually impaired understandthe situation more deeply. This can be consistent withNTT Technical Review

Vol. 18 No. 3 Mar. 2020

the notion that the processes of generating a sense ofbeing deeply moved (“Kandoh” in Japanese) are tension and release [2].
3.2 Judo
The translation of the judo match involved two
translators (sighted persons) and a visually impaired
person, and used a towel to express force interactions.
Two translators held both ends of a towel, and thevisually impaired person (with total late-onset blindness) grabbed the middle of it (Fig. 2). Translators
played the roles of the judo players by pulling thetowel up and down and left and right to express movements, pulling against each other, falling backwards
to destabilize the opponent’s center of gravity, etc.
The visually impaired person described his impression of it as like “another judo match was being heldon the towel. It felt more judo-like that watching it on
television.” The translations enabled some kind oftransformation or interpretation to occur, which gave
rise to other events while imitating the competition.
This is because the translators were able to reflect theanxiety, tension, surprise, and joy that they felt intheir translations. In other words, the visually
impaired persons were able to experience another’s
events through the translators’ interpretations, rather
than just simply being conveyed visual informationthat had been converted into haptic information. This
could be what led to the impression that “another judomatch was being held on the towel” described above.
This research project was inspired by “museum
60

Feature Articles

Fig. 2. Translation of judo match.

social view” in the field of art appreciation. In art
appreciation, people with visual disabilities andsighted people interact through dialogue in front of
art works to deepen their understanding of them.
Although art appreciation and sports watching may
seem different, both are similar in that they entail
deep understanding of the subject through the physicality and interpretation of others. Thus, we call our
project “Sports Social View.”
4. Essence and description in translation
Having translated sports for several competitions,
we realized that there are two factors in the expression of a translation. One is the sense of the essenceof movements that express the rhythm and impact ofa tennis rally or the destabilizing of the center of gravity in judo. The other factor is description. This is
information that conveys the state of a competition
such as the position of the ball and winning and losing. The translation is generally structured to put theessence on the description element. In tennis, thedescription element is expressed by tapping positions
to represent the ball being hit in a rally, while theintensity and way of tapping express the impact of theplayer hitting the ball as the essence. In tennis or badminton, although the description elements that
express the space in rallies are similar, the essences ofthe ball bouncing in tennis and the shuttlecock cutting
through the air in badminton are different.
If we review sports from the perspective of essence
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Fig. 3. Translation of table tennis (Photo by K. Nishida).

and description elements, we can reclassify themfrom different perspectives. Hence, we are currently
working on further deepening translations by asking
researchers specializing in sports and sports players
about the essence of competitions. Activities of thisproject have been published on the “Invisible Sports
Picture Book” [3] website, which currently introduces various sports such as rugby, table tennis, andsailing. In the case of table tennis, although the speed
and force of hitting seems important from the viewpoint of an amateur, we found “reading the rotation of
the ball” is rather important, so rotation is expressedby hitting a disk (Fig. 3). Soccer and rugby are also
competitions that develop around the ball in a similar
61

Feature Articles

Fig. 4. Tennis match experienced in darkness.

wide-open space, but what is different is that soccer
gets its attraction from the development of tactics,
whereas rugby is highlighted by a battle of players
trying to steal the ball from each other as they collide
and the forcefulness of that activity. Thus, pursuingthe essences of sports that involve fighting for balls
entails significant differences. Even if someone is asighted person, one could say that he/she could not“see” the sport. Thus, watching sports with translated
expressions of essences such as those above may beeffective not only for the visually impaired but also
for sighted people.
5. Technology for sharing inclusive sports
viewing experiences
Through this kind of sports translation, we wanted
to share the experience of sports translation with evenmore people across time and space as the principlethat the visually impaired and sighted can enjoy
watching sports together became more or less established. We used tactile recording and presentation
technologies to accomplish this aim. Focusing on the
sports translation of tennis in particular, we produced
exhibitions of “tennis match experienced in darkness” and “tennis match felt through the palm”
through which one can experience sports translation
anywhere and anytime via recording and playback ofvibrations.
5.1 Tennis match experienced in darkness“Tennis match experienced in darkness” involves a1800 x 900 mm table that mimics a tennis court.
Sports translations are carried out on the table, then
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

participants experience a tennis match by touching
the table (Fig. 4). The translator taps the table in sync
with the tennis rally, and microphones are attached tothe underside of the table at four locations to record
the vibrations. Vibration speakers are attached at thesame positions as the microphones, and the recorded
vibrations are replayed with the video. Participants inthis exhibit place their palms on the table and feel thevibration of the table being tapped. The experience
consists of the above table and a large screen in asmall room, and a tennis match is viewed under thefollowing conditions. (1) Watching tennis with all
information; video, ambient sound, and vibrations(sighted-people situation); (2) enjoying tennis withthe video subtracted and only ambient sound andvibrations (vision subtracted); and (3) enjoying tennis
with video and sound subtracted, with only vibrations(vision and hearing subtracted). This makes it possible to create different physical situations and enjoy
sports through physical translation.
In interviews after the experience, the visually
impaired described their understanding of the matchas “it was the first time I could tell where the ball hit
in a tennis match” and described their feeling of unity
as “I felt like I was watching with someone.” Some ofthe sighted people described the essence of tennis as
having their “senses sharpened and feeling more therhythm of rallies,” which indicates that physicaltranslations are also an effective way for sighted
people to enjoy sports.
5.2 Tennis match felt through the palm
“Tennis match felt through the palm” involves a
450 x 300 mm table that mimics a tennis court. In the62Feature Articles

Fig. 5. Tennis match felt through the palm.

same way as “Tennis match experienced in darkness,”tennis video and sound are replayed with a smartphone and headphones, and tapping vibration is
replayed with vibrating speakers attached to theunderside of a table at four locations (Fig. 5). Participants watch a tennis video on their smartphone screens
while listening to the sound through headphones andfeel the player hitting the ball by placing their palms
on the table. The smartphone provides video andaudio playback as well as power. This is a miniatureversion of “Tennis match experienced in darkness”
that enables sports translation in stadiums and public
viewing sites where an external power source is notavailable. As described above, exploring the spectator
experience with the visually impaired also made it
possible to create new ways for sighted people toenjoy sports, thus opening up the possibility of an
inclusive spectator experience.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

6. Future developments
Our project is based on the idea of translating theessence of sports and sharing it with the visually
impaired. Although there are people with many typesof physical conditions, changing the modalities of
sensation might enable many people to enjoy sports
viewing together. In the near future, we would also
like to apply this method to actual situations such as
in stadiums and public viewing spaces.
References
[1]
[2]
[3]

D. Oshimi and M. Harada, “Heart-capturing (Kandoh) Scene inSports,” Japanese Journal of Sport Management, Vol. 2, No. 2, pp.
163–178, 2010.
A. Tokaji, “Mechanisms for Evoking Emotional Responses of ‘Kandoh (the State of Being Emotionally Moved)’,” Cognitive Studies,Vol. 8, No. 4, pp. 360–368, 2001.
Invisible Sports Picture Book (in Japanese), https://mienaisports.com/

63

Feature Articles

Akiko Hayashi

Researcher, NTT Service Evolution Laboratories.
She received a master’s degree from Graduate
School of Frontier Biosciences, Osaka University, and became a researcher at NTT Communication Science Laboratories in 2009. Her research
is focused on user experience design and cognitive science.

Kentaro Shimizu

Senior Research Engineer, Supervisor, NTT
Service Evolution Laboratories.
He became a researcher at NTT in 1999 andreceived a Ph.D. in medical science from theTokyo Women’s Medical University in 2009. He
studies medical image processing and artificial
intelligence, and his research interests are image
processing and communications.

Junji Watanabe

Senior Distinguished Researcher, NTT Communication Science Laboratories; NTT Service
Evolution Laboratories.
He received a Ph.D. in information science andtechnology from the University of Tokyo in
2005. From 2005 to 2009, he was a PRESTO
researcher in the project called Foundation ofTechnology Supporting the Creation of DigitalMedia Content at the Japan Science & Technology Agency (JST). He became a researcher at
NTT Communication Science Laboratories in2013 and concurrently at NTT Service Evolution
Laboratories in 2016. His research is focused on
cognitive science and communication devices
with applied human perception. His interests arevisual and haptic perception and communications.

NTT Technical Review

Vol. 18 No. 3 Mar. 202064

Regular Articles

Field Trials of Open andDisaggregated Transponder in
Optical and IP Networking
Seiki Kuwabara, Yasuhiro Mochida,
Takahiro Yamaguchi, and Hideki Nishizawa
Abstract

An open and disaggregated optical transmission system is a candidate enabler for future high-capacity,
low-latency, and flexible optical networks. The open software architecture is key to developing an opentransponder for this transmission system. In this article, we propose the new optical transponder
architecture. We conducted field trials of low-latency optical transmission using an open anddisaggregated transponder with the open software architecture in a datacenter interconnect network to
confirm the feasibility and usefulness of this transponder.
Keywords: whitebox, datacenter interconnect, TIP

1. Introduction
NTT proposed and is working on the Innovative
Optical and Wireless Network (IOWN), which iscomposed of the All-Photonics Network, Cognitive
Foundation ®, and Digital Twin Computing [1],resulting in an innovative network with excellent
capacity, low latency, flexibility, and energy efficiency to create a smart world. Focusing on thephysical layer, in addition to time-of-flight of light inan optical fiber and signal processing time at transmission ends, one of the bottlenecks for achieving
ultimate low latency might be a hierarchical tree-type
network topology with vertically integrated opticaltransmission equipment. There have recently been
serious discussions on an open and disaggregatedoptical transmission system, which can add more
flexibility to the network topology. This activity is
expected to drive the development of a wide range ofservices by providing openness and multi-vendor
implementation at the component level and enabling
the addition of user’s own applications and autonomous operation. It will also enable us to tailor network topology for each application, such as highNTT Technical Review

Vol. 18 No. 3 Mar. 2020

speed storage backup via a wide area network ortransmission of multi-channel uncompressed ultrahigh-definition images captured at a sporting event
via a network to a broadcast station. It is therefore
important to create a platform for accelerating flexibility and technical innovation in this optical transport network.
2. Activities towards open and disaggregatedoptical transmission systems
In the traditional optical transport network, opticaltransmission equipment is locked into specific vendors in a vertical integration model and fixed from thetransceiver to the operating system. This approach
results in high implementation and maintenancecosts, makes decentralization of supply sources difficult, and usually forces network interoperability on
the user. It also makes it difficult to tailor network
topology for applications. NTT laboratories havebeen developing flexible control technology for optical transport networks that facilitates the virtualization of ICT (information and communication technology) resources and started collaboration with its
65Regular Articles

Fig. 1. Full mesh network topology with whitebox packet transponder.

global partners including network operating system(NOS) vendors, original design manufacturing vendors, optical component vendors, and other partners
to define the open interface of an optical transponder
through activities in the Telecom Infra Project (TIP).
In December 2017, the first interface specification
and architecture of the Transponder Abstraction
Interface (TAI) [2], which facilitates disaggregation
between software and hardware of an optical transponder, was proposed to TIP. Its working model wasthen exhibited at TIP Summit 2018 and the Optical
Networking and Communication Conference &Exhibition (OFC) 2019 with open hardware [3]. Thissection describes an optical network adaptable tocloud-native systems that we have been studying andsoftware architecture for an open optical transponder.
2.1 O ptical network with open and disaggregatedoptical transmission systems
Conventional networks feature a hierarchical treetype topology based on the telephone network andInternet access. One key advantage of this topology isuser-manageability; it allows operators to handle andmaintain a huge number of calls or services to each of
their users simply with a hierarchical structure. On
the other hand, an optical transport network that
handles cloud services, Internet of Things, and edge
computing used to be required to support ultra-lowlatency communication machine to machine (M2M).
An issue with tree-type topology is the high latency
for M2M because communications between edges
when they are under different branches need to passup and down the tree through many optical fiber paths
and equipment units. The full-mesh network with awhitebox packet transponder shown in Fig. 1 enables
lower latency at low cost because of capital expendiNTT Technical Review

Vol. 18 No. 3 Mar. 2020

ture reduction with multi-vendor coherent module
capability, transponder-switch converged architecture, open hardware, and rich network function
implementation capability at the carrier edge withcloud-native and micro-service architecture.
2.2 T ransponder architecture for open opticaltransponder
Achieving an optical network adaptable to cloudnative systems, as described above, will require a
reduction in the cost of implementing and operating
an optical transport network as well as flexible network control. In datacenters, the evolution of open
source software (OSS) accelerates automation ofoperations such as zero-touch provisioning andstreaming telemetry. When the disaggregation of
hardware and software technology developed fordatacenter servers and switches spreads to the opticaltransport network field, operators can also use automatic operation technologies with OSS for opticaltransport networks. We propose the optical transponder architecture shown in Fig. 2 that adopts a Linuxbased NOS and enables the disaggregating of opticaltransmission equipment into hardware and software
in a whitebox format. This architecture provides
Ethernet-based optical and Internet Protocol (IP) networking capable of simple operation while adopting
packet switching integrated architecture.
It also hides the proprietary specifications for thecoherent components loaded on the optical transmission equipment and adopts the TAI [4], which is anapplication programming interface for vendor/generation-independent coherent module control. The useof the TAI reduces NOS development and maintenance costs and enables NOS and optical transmission equipment to be combined in diverse ways. In
66

Regular Articles

Vendor proprietary

Controller

Controller

Open software/hardware

North APISouth API

Transport manager
South API

Open switch manager

Open transportmanager

SAI

TAI

Switch SDKTransport SDK
Open Kernel

Operating system
Card API
Transport Chassis
FPGA

Open hardware

Line card
FPGA
DSP

Conventional transponder
API: application programming interface
ASIC: application-specific integrated circuit
DSP: digital signal processor

X86

SwitchASIC

DSPWhitebox packet transponder
FPGA: field-programmable gate array
SAI: Switch Abstraction Interface
SDK: software development kit

Fig. 2. Conventional and proposed transponder architectures.

(a) Cassini(b) Galileo

Fig. 3. Whitebox packet transponders.

addition, hardware in this architecture takes on a
modular configuration, which means that it can beused for small starts and easily exchanged as needed.
The whitebox packet transponders Cassini and Galileo shown in Fig. 3 are being proposed at TIP for an
open optical transport network [5].
3. Field trials of uncompressed video
transmission in commercial
datacenter interconnect
There is currently a major move toward the use of
IP for transmitting video and audio in broadcast
facilities. This transition is expected to enable
uncompressed video/audio transmission, making fulluse of the large-capacity, low-latency, and bidirectional communication characteristics. Interest is also
growing in developing a remote production system
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

that interconnects a relay site and broadcast station
over an IP network and sends the video and audio
acquired in the field to a broadcast station without
prior editing (Fig. 4). Since the editing process takes
place at the broadcast station, fully equipped production vans are no longer required and editing crews areallowed to stay at the broadcast station. However,multiple Ethernet switches are required for long-distance transmission, which can affect the Precision
Time Protocol used for synchronization among video
devices. Long-distance and large-capacity capabilities in the transmission of such video and audio are
therefore important. Taking this into account, we conducted field trials of 4K/60P video using commercial
dark fiber between datacenters to assess the effectiveness of our proposed optical transponder architecture(Fig. 5). In these field trials, we used Cassini whitebox packet transponders to directly connect two
67

Regular Articles

Shooting

Editing

Transmission
Transmitter

ReceiverEditing crews

IP network

Fig. 4. Remote production workflow.

Receiving side

Transmitting side
4K video
reproducing
deviceTransmitter
CassiniTransmitter
3G-SDI x4

100G
20 km

Cassini

Receiver4K monitor
Receiver3G-SDI x4

Fig. 5. Experimental setup of large-capacity transmission of uncompressed 4K video.

points over approximately 20 km of dark fiber. For
optical transmission, we used an analog coherent
optics transceiver for carrying out digital coherent
processing on the digital signal processor mounted ona Cassini plug-in unit. This pair of Cassini transponders is equipped with different NOSs, uses quadrature phase-shift keying as the modulation system, andinterconnects by 100-Gbit/s Ethernet.
The 4K/60P video to be transmitted is output froma 4K video-reproducing device using four 3 Gbps
Serial Digital Interface (3G-SDIs). These video
streams are then input to two transmitters—two
3G-SDIs per transmitter—and the outputs from thesetransmitters are finally output as separate SMPTE ST
2110* flows. These transmitters connect directly to a
Cassini transponder. The receivers used in this trial
likewise connect to a Cassini transponder and thereceived IP flows are output by the four 3G-SDIs.
Using this system, we conducted the field trials from
July to September 2019 (Fig. 6). These trials confirmed that the video could be correctly reproducedon the receiver side and that 4K/60P uncompressedvideo could be transmitted using Cassini transponders.

Fig. 6. Transmission of 4K video to receiver side.

ble control of the optical network and easy additionof services. Going forward, we plan to develop technology for autonomously controlling and stabilizing
the optical network in response to changes in theenvironment, application operations, etc.

4. Conclusion
The optical transponder architecture we proposed is
adaptable to cloud-native systems and enables flexiNTT Technical Review

Vol. 18 No. 3 Mar. 2020*

SMPTE ST 2110: A standard developed by the society of motion
picture and television engineers (SMPTE) for transmitting video
over IP networks for the professional media industries.

68Regular Articles

References
[1]

[2]

S. Iwashina, Y. Aragane, K. Minamihata, K. Shindo, and M. Fujiwara,“Initiatives to Realize the IOWN (Innovative Optical and Wireless
Network) Concept,” NTT Technical Review, Vol. 18, No. 2, pp.
27–31, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202002iw1.html
M. Long, “Open Optical Packet Transponder Leveraging OCP
Networking Technology,” OCP Summit 2018, San Jose, CA, USA,
Mar. 2018.

Seiki Kuwabara

[3]

[4]
[5]

https://www.youtube.com/watch?v=krOsB7va9M4
H. Schmidtke and H. Nishizawa, “TIP at OFC 2019: Collaboration
with ONF, OOPT Project Group Updates,” Mar. 2019.
https://telecominfraproject.com/tip-at-ofc-2019-collaboration-withonf-oopt-project-group-updates/“Disaggregated Transponder Chip Transport Abstraction Interface,”https://github.com/Telecominfraproject/oopt-tai
V. López and H. Schmidtke, “Prime Time for Open and Disaggregated
IP and Optical Technologies,” Nov. 2019.
https://telecominfraproject.com/prime-time-for-open-anddisaggregated-ip-and-optical-technologies/

Takahiro Yamaguchi

Research Engineer, Photonic Transport Network Laboratory, NTT Network Innovation
Laboratories.
He received a B.E. and M.S. from Osaka University in 2007 and 2009. Since April 2009, hehas been with NTT Network Innovation Laboratories. His current research interests include
optical transport network operation systems andwhitebox packet transponder switches. He is amember of the Institute of Electrical and Electronics Engineers/Communication Society.

Group Leader, Senior Research Engineer,
Supervisor, Media Innovation Laboratory, NTT
Network Innovation Laboratories.
He received a B.E., M.E., and Ph.D. in electronic engineering from the University of Electro-Communications, Tokyo, in 1991, 1993, and1998. He joined NTT in 1998 and has been
researching super high definition image distribution systems. He is currently a group leader of themedia processing systems research group in NTT
Network Innovation Laboratories. He is a member of IEICE and the Institute of Image Information and Television Engineers.

Yasuhiro Mochida

Hideki Nishizawa

Research Engineer, Media Innovation Laboratory, NTT Network Innovation Laboratories.
He received a B.E. and M.E. from The University of Tokyo in 2009 and 2011. Since he joined
NTT in 2011, he has been engaged in research onvideo transmission over IP networks includingtransport protocol, presentation synchronization,and video conferencing. His current researchinterest is in low-latency video transmission overhigh-speed optical networks. He is a member of
the Institute of Electronics, Information andCommunication Engineers (IEICE).

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Senior Research Engineer, Supervisor, Photonic Transport Network Laboratory, NTT Network Innovation Laboratories.
He received an M.S. in physics from Chiba
University in 1996. He joined NTT in 1996,
where he is engaged in research on ubiquitous
networking, photonic switching systems, andphotonic transmission systems. He is a memberof IEICE.

69

Regular Articles

Does Exploratory Hand Movement
Provide Information on Tactile
Sensations?
Takumi Yokosaka
Abstract

People move their hands in a variety of ways, such as tracing a finger on a material to feel its roughness
or pushing a materials’ surface to feel its hardness, to judge the feel of materials around them. Many
studies on human tactile sensation did not focus on such hand movements. This article introduces mystudies on investigating whether hand movements—and even eye movements—can be used to provide
information for estimating the tactile sensation felt by a person. These studies are expected to provide
not only important information for clarifying the mechanism of tactile perception but also help indesigning guidelines for tactile-information display devices attached to arms and hands.
Keywords: human perception, tactile perception, exploratory hand movement

1. Introduction
People can feel a variety of tactile sensations (e.g.,hardness and roughness) by actively exploring thesurface of an object with their hands and fingers.
While haptic-display technology and product-surface
design technology, which are applied to induce sucha predetermined tactile sensation, have been attracting attention, how these technologies actually create
an end-user tactile sensation has received little attention. In fact, even if many people touch the same
object, they will not all feel the same tactile sensation.
For example, how surface roughness is perceived varies from person to person [1], and the perceived
roughness can change as the force of the fingers pushing the surface changes [2, 3]. Accordingly, even if
people are using the same haptic display or touching
the surface of the same object, the tactile sensation
felt by each person is likely to change from person toperson. One way to determine what type of tactile
sensation a person actually felt is to actually ask
them; however, from the viewpoint of an end user,
having the person report their tactile sensation every
time they touch something is inconvenient. It would
therefore be useful if it were possible to estimate
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

externally—by using an objective measurement
index—how a person felt when touching an object.
This article focuses on exploratory movement of ahand while touching an object as a source of information (i.e., a measurement index) about a tactile sensation. In many previous tactile studies, such hand
movements were treated as something that should becontrolled, and the tactile ability of a stationary handand a hand performing simple movements (such as
linear ones) were investigated. However, they overlooked the role of hand movements. It is known thathow hands move on objects changes according towhat type of tactile sensation is to be determined [4],and it is suggested that the movement of the handplays the role of a window that reflects the tactile
sensation. This article introduces my efforts to investigate how exploratory hand movements provide
information about tactile sensation.
2. People know the relationship between
exploratory hand movements and tactile sensation
Do exploratory hand movements actually containtactile information? If so, we might be able to use that
information. In other words, by looking at the70Regular Articles

Rough-smoothe.g. Skin after shaving

Hard-softe.g. Towel after washing

Sticky-slippery
e.g. Plate after washingFig.1. TV commercials use hand movements to convey a rich sensation of touch to viewers.

hand movements of someone else touching an object,
we might be able to imagine how that object feels
when touched. Although this situation may seem
strange, we are already familiar with it through television (TV) commercials (Fig. 1). For example, wehave seen scenes in which skin is stroked by a hand,a towel is pushed by a hand, or a plate is rubbed witha finger. Watching such scenes, we can imagine thesmoothness of the skin after shaving, the softness ofa towel washed using softener, and the frictionalsqueak of greasy plates that have been washed using
detergent. This imaginative ability of people was previously investigated through psychophysical experiments [5].
First, six participants (each called a toucher)touched an object (such as wood or glass) and evaluated the tactile sensation they felt. The hand movements of each toucher were recorded while he/she
was touching his/her target object. Ten other participants (each called an observer) were asked to observe
the touchers’ hand movements and guess how thetouchers felt when they touched the target object. To
investigate whether hand movement is actually used
as an information source, a toucher’s hand movement
was reproduced with moving light points that exclude
information other than hand movement (e.g., skindeformation and nail color) (Fig. 2). The experimental results indicated that all ten observers responded
in a similar manner. That is, when the observers
watched the light-point hand movements of a toucher
touching an object that the toucher felt was sticky,
they tended to judge that the toucher was touching a
sticky object. When the observers watched the lightpoint hand movements of a toucher touching an
object that the toucher felt was fluffy, they tended tojudge that the toucher was touching a fluffy object.
The results indicated that people can use a common
strategy—based on hand movements—for estimatingtactile sensations. It is thought that people possess
such a common strategy because the movement ofhands touching objects contains information on thetactile sensation felt by the toucher.
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Fig. 2. A scene from a video of hand movements of atoucher in a previous experiment [5]. The hand
movements were drawn as moving light points andobserved by observers.

3. Analysis of exploratory hand movements
What exploratory hand movements actually containcertain tactile information? An experiment was conducted in which participants were asked to touch
various objects that they encounter on a daily basis
and report their tactile sensations [6]. By analyzing
the measured hand movements, what movements can
explain these tactile sensations were determined. The
experimental results revealed that the speed of a
stroking hand and the magnitude of the hand’s pushing force contained information on the hardness,roughness, stickiness, and warmth felt by the participants. Two trends were identified: (i) the evaluationsof hardness and temperature (related to pushing andstatic contact) are related to the magnitude of the71Regular Articles

Correlation coefficientwith saccade motion

1.0

0.5

Cold

SmoothHard

Slippery0
Sticky
−0.5

−1.0
−1.0

Soft

Rough

−0.5

0

Warm

0.5

1.0

Correlation coefficientwith stroke motion

Fig. 3. Correlation between active movements of the eyes and hands and evaluation of tactile sensation. For example, active
eye movements (saccade motion) have a positive correlation with coldness, which indicates that the eyes move more
actively when someone is touching an object judged as cold. On the contrary, the stroking motion of the hand has anegative correlation with coldness, which means that the hand moves more actively when someone is touching an
object judged as warm.

pushing force and (ii) the evaluation of roughness andstickiness (related to surface friction) is related to the
speed of the stroking hand. Even more interesting isthat active eye movements also contain tactile information. The reason for this is still open to question,but exploratory eye movements might be related tothe physical properties of the object surface.
To estimate tactile sensations, it may be useful tocombine measurements of hand speed and active eye
movements. As shown in Fig. 3, active eye movements contain information on hardness, warmth, androughness, and the speed of the hand contains information on stickiness, roughness, and warmth. Fromthese results, it might be possible to specify which of
the four main touch sensations—hardness-softness,roughness-smoothness, stickiness-slipperiness, andwarmth-cold—are felt. For example, when a person
moves his/her eyes and hands, he/she is feeling asmooth sensation (first quadrant in Fig. 3), and whena person is not moving his/her eyes but moving his/her hands, he/she is feeling a warm sensation (fourth
quadrant in Fig. 3).
4. From perceived physical properties toevaluated preference
Hardness, roughness, stickiness, and warmth aretactile sensations that are closely related to the physiNTT Technical Review

Vol. 18 No. 3 Mar. 2020

cal properties of an object. For example, perceived
hardness is strongly affected by compliance [7, 8],
and perceived roughness is strongly affected by friction [9–11]. Perception of these sensations is necessary for properly adjusting movements, such as whena person is gripping an object. However, tactile sensation has not only this function but also a function that
determines how favorable the touched object is to theperson touching it. Differing from sight and hearing,tactile sensation is a perception that occurs when the
body is directly in contact with an object, so thisevaluation of tactile preference is important. I also
investigated whether movements of a hand touchingan object also contain this perception of preference[12]. First, I showed that the evaluation of preference
can change if the manner of touching changes. For
example, woody materials were evaluated as preferable when being stroked but evaluated as less preferable when being pressed. This means that there is a
close relationship between how an object is touched
and tactile sensation (preference evaluation). Next, I
discussed the analysis of the movement of a handtouching an object and that the features related to
stroking motion (such as hand speed) contained
information on the evaluation of preference. On thecontrary, the features related to pushing did not contain such information. These results suggest thatmeasuring stroking motion will allow us to estimate
72Regular Articles

evaluated preference.

felt by the wearer.

5. Benefits of measuring hand movements andapplying the measurement results
According to the results presented thus far, it may
be possible to estimate how a person feels by measuring his/her hand movements. Therefore, what are thebenefits of measuring and analyzing hand movements
and how does this differ from measuring and analyzing physical properties (such as compliance and friction)? One benefit of measuring hand movements ismaking it possible to consider the effects of individual differences and manner of touching, as described in
the introduction. For example, individual differences
due to differences in search strategies of end users
cannot be determined by measuring the physicalproperties of touched objects; however, they might bedeterminable by measuring hand movements. In fact,which information is used preferentially, spatial
information or temporal information regarding theperception of roughness, depends on the person [1].
Since it is also known that hand movements differ
according to the manner in which such spatiotemporal information is used [13], measuring such movements might allow us to consider individual differences. As stated above, the tactile sensation (evaluation of roughness or evaluation of preference) canchange when the manner of touching an objectchanges even if the same person touches the object.
This fact cannot be understood from measuring thephysical properties of the object; instead, it could beunderstood by measuring hand movements. To analyze the physical properties of objects, we must measure the physical properties of all objects end users
might touch. To analyze hand movements, however,
all we need to do is attach a motion sensor to thehand. This approach has a high affinity with a haptic
display worn on the arm; that is, attaching such a
device to measure the movement of the arm makes it
possible to continuously monitor the tactile sensation

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

6. Conclusion
Studies on whether hand movements contain information on tactile sensation were introduced. The
results of these studies are expected to contribute to
not only establishing an objective method for estimating tactile sensation but also revealing human-tactile
mechanisms that are still unknown.
References[1]

M. Natsume, Y. Tanaka, and A. M. L. Kappers, “Individual
Differences in Cognitive Processing for Roughness Rating of Fine andCoarse Textures,” PLoS One, Vol. 14, No. 1, e0211407, 2019.
[2] S. J. Lederman and M. M. Taylor, “Fingertip Force, Surface
Geometry, and the Perception of Roughness by Active Touch,”
Percept. Psychophys., Vol. 12, No. 5, pp. 401–408, 1972.
[3] S. J. Lederman, “Tactile Roughness of Grooved Surface: The
Touching Process and Effects of Macro- and Microsurface Structure,”
Percept. Psychophys., Vol. 16, No. 2, pp. 385–395, 1974.
[4] S. J. Lederman and R. L. Klatzky, “Hand Movements: A Window into
Haptic Object Recognition,” Cogn. Psychol., Vol. 19, No. 3, pp.
342–368, 1987.
[5] T. Yokosaka, S. Kuroki, J. Watanabe, and S. Nishida, “Estimating
Tactile Perception by Observing Explorative Hand Motion of Others,”
IEEE Trans. Haptics, Vol. 11, No. 2, pp. 192–203, 2018.
[6] T. Yokosaka, S. Kuroki, J. Watanabe, and S. Nishida, “Linkage
between Free Exploratory Movements and Subjective Tactile
Ratings,” IEEE Trans. Haptics, Vol. 10, No. 2, pp. 217–225, 2017.
[7] R. Harper and S. S. Stevens, “Subjective Hardness of Compliant
Materials,” Q. J. Exp. Physiol., Vol. 16, No. 3, pp. 204–215, 1964.
[8] S. Wongsriruksa, P. Howes, M. Conreen, and M. Miodownik, “TheUse of Physical Property Data to Predict the Touch Perception ofMaterials,” Mater. Des., Vol. 42, pp. 238–244, 2012.
[9] L. A. Stone, “Subjective Roughness and Smoothness for Individual
Judges,” Psychon. Sci., Vol. 9, No. 6, pp. 347–348, 1967.
[10] R. T. Verrillo, S. J. Bolanowski, and F. P. McGlone, “Subjective
Magnitude of Tactile Roughness,” Somatosens. Mot. Res., Vol. 16,
No. 4, pp. 352–360, 1999.
[11] G. Ekman, J. Hosman, and B. Lindstroem, “Roughness, Smoothness,
and Preference: A Study of Quantitative Relations in Individual
Subjects,” J. Exp. Psychol, Vol. 70, pp. 18–26, 1965.
[12] T. Yokosaka, M. Inubushi, S. Kuroki, and J. Watanabe, “Frequency of
Switching Touching Mode Reflects Tactile Preference Judgment,”
Sci. Rep., Vol. 10, 3022, 2020.
[13] E. Gamzu and E. Ahissar, “Importance of Temporal Cues for Tactile
Spatial-Frequency Discrimination,” J. Neurosci., Vol. 21, No. 18, pp.
7416–7427, 2001.

73

Regular Articles

Takumi Yokosaka

Scientist, Sensory Representation ResearchGroup, Human Information Science Laboratory,
NTT Communication Science Laboratories.
He received a B.E. and M.S. in engineering andinformation science from Osaka University in2011 and 2013, and a Ph.D. in information processing from the Tokyo Institute of Technology
in 2018. His research interests include perception
and cognition shaped by body movement.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

74

Global Standardization Activities

Activities at W3C Technical Plenary
and Advisory Committee Meetings
Week (TPAC) 2019 in Fukuoka
Yumiko Matsuura, Kiyoshi Tanaka, Shigeru Fujimura,and Koichi Moriyama
Abstract

The W3C (World Wide Web Consortium) Technical Plenary and Advisory Committee Meetings Week
(TPAC) 2019 was held from September 16 to 20 in Fukuoka, and this meeting was the second time heldin Japan. The activities of the NTT Group related to development of web technology in TPAC aredescribed in this article.
Keywords: W3C TPAC, web authentication, digital signage

1. Overview of W3C TPAC 2019 in Fukuoka
The Technical Plenary and Advisory Committee
Meetings Week (TPAC) is an annual general meeting
held by the World Wide Web Consortium (W3C),
which is a membership-based international consortium of industrial and academic organizations
involved in standardizing and promoting web technology. The days before and after the plenary session
held on that Wednesday were filled with various
activities related to standardization, including faceto-face technical discussions in the Working Group(WG) and discussions on the use of technology froma business perspective and technology that has been
adopted in the Business Group (BG). TPAC 2019 washeld at the Hilton Fukuoka Sea Hawk and was thelargest ever, with over 640 participants and over 100
meetings and sessions.
The key topics of the plenary session included anannouncement regarding Sir Tim Berners-Lee, inventor of the World Wide Web as well as the founder andcentripetal force within the W3C, of his intention to
step down as director. There was also a report on the
proposal for the W3C to obtain legal entity status inJanuary 2021, which has been discussed in various
meetings as a means to achieve stable operation of the
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

W3C, and members were asked to make preparations
within their own organizations. Concerning the handling of intellectual property in the standardization
process, a proposal for a policy of early confirmationof royalty-free implementation by each organization,
which is currently required at the time of recommendation, was promptly confirmed to prevent reworking
in WG discussions.
2. NTT Group activities
The NTT Group provided stable Internet connection service during the conference via NTT Communications and NTT WEST. Meeting participants
included many individuals from many countries, andcommunication between members via the Internet forweb conferencing, GitHub, etc. required a very fastand robust network environment. By continuously
and reliably performing daily tasks such as monitoring during meetings and equipment inspection after
meetings, the NTT Group contributed to the highevaluation of the meetings by participants.
3. FIDO authentication and the WebAuthn WG
The Web Authentication (WebAuthn) WG has been75

Global Standardization Activities

standardizing a JavaScript application programming
interface (API) for web browser implementation of
simple and strong web authentication based on the
FIDOTM*1 model [1, 2].
The standardization of web authentication in the
W3C began with the contribution of the basic specifications of the web part (draft) by the FIDO Alliancein November 2015. The FIDO certification model
was made compatible with operating systems andbrowser platforms and officially recommended as
Level 1 in March 2019. Level 2 is currently being
formulated.
NTT DOCOMO and NTT laboratories joined the
FIDO Alliance as a board member and a sponsor
member, respectively. NTT DOCOMO has chaired
the Consumer Deployment WG and FIDO Japan WG
and is contributing to the formulation and popularization of FIDO certification specifications by providing
summaries of issues and feedback regarding commercial introduction and improvement of the FIDO
specifications.
Seizing the opportunity of getting together with key
persons involved in web security and authentication
at TPAC 2019, three initiatives from Japan were taken
up to promote even more widespread application of
the simple and robust FIDO authentication model incommercial services, including web authentication.
3.1 Demonstration booths
The NTT Group took the initiative to set up web
authentication demonstration booths during the weekof the meeting. The FIDO Japan WG and the JapanFIDO Alliance member companies also participatedby demonstrating web authentication on smartphones, web authentication using personal computersand security keys, and the operation of an authentication device that uses the veins in users’ palms. By
popularizing FIDO certification and introducing new
first-in-the-world initiatives in Japan, we were able todemonstrate the appeal of password-less authentication.
3.2 Luncheon speech
On the second day, we conducted a 30-minute presentation entitled “Contributions from NTT andJapan Teams for Simpler, Stronger Authentication.”
The presentation covered how NTT DOCOMO wasan early adopter by using the FIDO specifications ford ACCOUNTTM*2 log-in biometric authentication
since May 2015 [3] and initiatives of the NTT Groupand Japanese companies regarding web authentication. Knowledge gained from world-first commercial
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

applications and feedback for improving specifications were also presented along with ideas for moving
forward. At TPAC 2019, which was held in Japan for
the first time in a long while, a luncheon speech (afirst for TPAC) describing valuable Japanese initiatives generated strong interest and lively discussion
among several audience members (Photo 1).
3.3 WebAuthn WG
In the WebAuthn WG, more specific feedback wasoffered, and there was discussion on formulating
Level 2. The three main points were 1) current differences in browser implementations, 2) problems withthe specifications for providing a frictionless authentication experience, and 3) best practices for effective
use of the many options for implementing passwordless authentication. We believe these efforts haveincreased the momentum of standardization activities
for reducing the risk of unauthorized access originating in the online use of passwords.
4. Web-based Signage BG
Web-based signage [4] is digital signage that uses
web technology and features the implementation ofservices by simple installation of a web browser on aterminal. In the W3C, the Web-based Signage BG(co-chaired by NTT) began studies on the implementation of web-based signage in April 2012, startingwith analysis of use cases, and has been discussing
the implementation of the browser API required byservices. The Web-based Signage BG meeting washeld at TPAC 2019 for the first time in about two
years. The history of the activities was reviewed, andit was confirmed that web-based signage has alreadybeen commercialized and is in use worldwide. Having achieved the initial goal of widespread use of
web-based signage, the members agreed at the meeting to terminate activities of this BG.
5. Breakout session topics
One of the features of TPAC is “unconferencestyle” breakout sessions held on the day of the plenary session. Volunteer participants deal with allmatters from setting the discussion topics to managing the sessions. The discussion topics can be very
diverse, such as seeking broad audience perspectives*1 FIDO stands for First IDentity Online. FIDO is a trademark of
FIDO Alliance.
*2 d ACCOUNT is a trademark of NTT DOCOMO, INC.

76

Global Standardization Activities

Photo 1. Luncheon speech at the TPAC.

on divided opinions within the WG, or proposing new
topics for future discussion in the W3C. This time, 59
sessions were held with lively discussion, and up to
12 sessions were conducted simultaneously.
Of particular interest was the six related sessions
presented by Google and Apple regarding browser
privacy, including increased security risks and proposals for new browser security models. Behind this,IP (Internet protocol) addresses and cookies used for
personalization in web advertising are considered
personal information under the General Data Protection Regulation (GDPR)*3 and require careful handling. Another background issue is the considerable
progress in browser fingerprinting [5] as a method of
tracking users that does not use cookies and is moredifficult to prevent.
It goes without saying that the protection of privacy
is an important issue, and browser vendors are

expected to propose new specifications based on thediscussions held in these sessions in the near future.
References
[1]
[2]
[3]

[4]
[5]

FIDO Alliance, https://fidoalliance.org/
WebAuthn WG, https://www.w3.org/TR/webauthn/
K. Moriyama, “Toward a Password-free World: NTT DOCOMO
Efforts on d ACCOUNT Biometric Authentication Using the FIDOStandard and Future Prospects,” Denkitsushin (Telecommunications),
Vol. 80, No. 840, pp. 13–19, 2017 (in Japanese).
K. Tanaka, M. Nakamura, K. Suzuki, and K. Takegami, “Trends in
Web-based Signage Standardization,” NTT Technical Review, Vol.
15, No. 8, 2017.
N. Nikiforakis, A. Kapravelos, W. Joosen, C. Kruegel, F. Piessens, and
G. Vigna, “Cookieless Monster: Exploring the Ecosystem of Webbased Device Fingerprinting,” Proc. of the 2013 IEEE Symposium on
Security and Privacy, pp. 541–555, Berkeley, USA, May 2013.

Trademark notes

All brand, product, and company/organization names that appear in thisarticle are trademarks or registered trademarks of their respective owners.

*3 GDPR: European Union (EU) regulation for general data protection formulated by the European Parliament, European Council,
and European Commission intended to strengthen and integrate
data protection for all individuals in the EU.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

77

Global Standardization Activities

Yumiko Matsuura

Senior Research Engineer, Supervisor, Universal UX Design Project, NTT Service Evolution
Laboratories.
She received a B.S. and M.S. in computer science from Keio University, Kanagawa, in 1991
and 1993. She joined NTT Human Interface
Laboratories as an engineer in 1993 and engaged
in the development of multimedia systems and acontent delivery platform. She was in the
Research and Development Strategy Department
twice, most recently she started collaboration
with leading companies in industries such as
automotive and manufacturing. As a result oforganizational changes in July 2017, she is now
at NTT Service Evolution Laboratories and is
involved in research management on humanunderstanding and interface design.

Kiyoshi Tanaka

Senior Research Engineer, Supervisor, Coinnovation Promotion Project, NTT Service
Evolution Laboratories.
He received a B.E., M.E., and Ph.D. in communication engineering from Osaka University,
in 1992, 1994, and 2005. He joined NTT in 1994
and since then has been engaged in researching
video-on-demand systems and metadata-related
interactive video systems and services, especiallythose related to IPTV and digital signage services. His current interests include standardizationof digital signage. He is one of the co-chairs of
the W3C Web-based Signage Business Group.
He is also a member of the Institute of Electronics, Information and Communication Engineers(IEICE), the Human Interface Society in Japan,
and the Institute of Image Electronics Engineersof Japan (IIEEJ).

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Shigeru Fujimura

Senior Research Engineer, NTT Service Evolution Laboratories.
He received an M.S. in information science andtechnology from the University of Tokyo in 2005
and joined NTT the same year. Since then, he has
been engaged in research on web mining and web
engineering, especially on effective methods of
implementing web applications. His researchinterest is data management systems using blockchain and smart contract technology.

Koichi Moriyama

Senior Director of Product Innovation, ProductDepartment, NTT DOCOMO, INC.
He received a B.E. in electrical engineeringand an M.E. in computer science from Keio
University, Kanagawa, in 1992 and 1994. He is
responsible for the innovation of NTT DOCOMO’s devices and platform software and for thepartnerships that support NTT DOCOMO’s
innovation goals, including those related to
FIDO standards. Mr. Moriyama has previously
held positions with Sony Ericsson Mobile Communications, Inc. and Sony Corporation. He wasalso a visiting research scholar at Georgia Institute of Technology, USA, while he was at Sony
Corporation.

78

Short Reports

NTT and Telefónica Start PoC on
Open Optical and Packet Transport
Technologies within Telecom Infra
Project

1. Introduction
NTT and Telefónica announced that they have conducted a successful proof of concept (PoC) on open
optical and packet transport technologies at the TIP
Summit’19 held in November 2019 in Amsterdam,
the Netherlands. A whitepaper of the PoC was published on the website of the Telecom Infra Project(TIP) in January 2020 [1].
The goal of the PoC is to implement open and commoditized networks to achieve wide scale economy
for some components of the Converged Architectures
for Network Disaggregation & Integration (CANDI)
architecture and create an open market for disaggregated elements, both Internet protocol (IP) and optical.
To achieve this goal, NTT and Telefónica created
the CANDI sub-group under the framework of TIP’s
Open Optical and Packet Transport (OOPT) projectgroup in October 2018. In 2019, Orange, Telia Company, and the pan-European research network operator GÉANT joined CANDI. Only one year after thecreation of the working group, the first collaboration
PoC was completed in Madrid, Spain.
With this PoC, the team succeeded in building a
converged network with a packet technology controlling telecommunication traffic and an optical technology transmitting data long distance, following therequired architecture, open technology and open
interface promoted by CANDI.
2. NTT’s MSF and Beluganos
development history
NTT has been engaged in the Multi-Service Fabric(MSF) project since 2014 to promote the use of open
NTT Technical Review

Vol. 18 No. 3 Mar. 2020

hardware provided by such communities as the Open
Compute Project (OCP), leading to disaggregation
between software and hardware in packet transport
equipment and the network architecture and developing its system. This project is enhancing service reliability and flexibility of service development, as
required by telecom operators. Since 2016, NTT has
also started developing in-house software, Beluganos,
which is a network operating system (OS) using an
open interface controlling packet transmission chip ina white-box switch to activate the functions that are
implemented by vendors (Fig. 1).
NTT has been disclosing both MSF and Beluganos
as open source software (OSS) respectively to accelerate the development of open technology in thepacket transport field and implementation of carrierrequired features by multiple vendors to avoid vendor
lock-in since 2017. In 2019, NTT contributed to andsucceeded in a PoC through collaboration with Taiwan’s largest carrier Chunghwa Telecom by using
these OSS.
3. Roles of Telefónica and NTT
To develop the PoC, Telefónica provided a multilayer controller based on open standard interfaces
with both the packet controller and optical controllers, and NTT provided an IP software-defined networking (SDN) controller (MSF) and network OS
Beluganos as the reference model of the architecture
and open interface (Fig. 2). Beluganos controls the
OpenNSL (Open Network Switch Library) as anopen interface of the packet transform chip of awhite-box switch and also provides Netconf/Openconfig, which is becoming the de-facto open interface
for remote-controlling packet devices. MSF succeeded79

Short Reports

Commercial products

REST

In-house software

MSF controller
NTT Network Service
Systems Labs.

Open I/F
Device vendor agnostic

NETCONF

VendorswitchSNMP

Network OS(Third party)(merchantsiliconor vendorspecific silicon)

Uniformly control multiple devices
manufactured by different vendors

NETCONF

NETCONF
SNMP

SNMP

Common GUI

MSF

Network OS(Beluganos)
NTT Network Service
Systems Labs.

API (OpenNSL)

APIWhite-box switch

Efficient implementation

White-box switch

Modification of eachfunction is possible.

API: application programming interface
GUI: graphical user interface
I/F: interface
NETCONF: Network Configuration Protocol

OpenNSL: Open Network Switch Library
REST: Representational State Transfer
SNMP: Simple Network Management Protocol

Fig. 1. MSF and Beluganos.

PoC Location: Telefónica Lab (Madrid)

NTT labs. (Tokyo)

Remarks

Remote testing

Telefónica

• Carrier’s softwareor OSS

Multi-layer controller(Metro-Haul project)

VPN

REST

TAPI (ONF)• Vendor product

MSF

Hardware

NETCONF/
OpenConfig

CLI

Optical controller(ONF: ONOS)NTT

TAPI (ONF)

NETCONF/OpenConfig

Packet controller
OSS (MSF)
CLI

Software
NetworkOS

Network OS(TIP)

Controller(ONF)

ClosedI/FTAI (TIP)

ClosedI/F

CassiniOpen line
system(ONF)

Whitebox(OCP)
Transponder(ONF)

Network OS(TIP)

Network OS
OSS (Beluganos)

NTT

NetworkOS

TAI (TIP)

OpenNSLClosedI/FWhite-box
transponder(TIP: Cassini)Whitebox(OCP)

Whitebox

Transponder(ONF)

Whitebox

Whitebox
ServerServerMulti vendor

CLI: command line interface
ONF: Open Networking Foundation
ONOS: Open Network Operating System

Single vendor

Multi vendor

TAI: Transponder Abstraction Interface
TAPI: Telephony Application Programming Interface
VPN: virtual private network

Fig. 2. Architecture of first PoC.

NTT Technical Review

Vol. 18 No. 3 Mar. 202080

Short Reports

in controlling three different network OSs on whitebox switches and controlling the open interface thatis Netconf/Openconfig for Beluganos.
This PoC is supported by Edgecore Networks,
ADVA, IP Infusion, and Infinera, as TIP members.
Open community, Open and Disaggregated Transport
Network (ODTN) in Open Networking Foundation(ONF), and the Metro-Haul project funded by European Union provided implementation and support in
the field of optical disaggregation.
4. Future prospects
Through this PoC, NTT and Telefónica have started
providing the feasibility of use cases required by theoperators in CANDI. At the same time, they started acommunity to clarify the issues to be addressed for
obtaining all the functions required by the use cases

including scalability and operational issues.
NTT and Telefónica will continue to contribute to
conducting enhanced PoCs twice a year in CANDI toprove such feasibility and find issues with developing
the technology required to confirm the feasibility ofoperators’ use cases. By maximizing the advantage ofdisaggregation technology and introducing the latest
technology much earlier, NTT and Telefónica willcontribute to avoiding vendor lock-in and enablingtelecommunication networks with which people canuse much more varied services faster.
Reference[1]

Website of NTT, “Telecom Infra Project has published the whitepaper of the collaboration PoC among NTT, Telefónica and otheroperators,”
https://www.ntt.co.jp/topics_e/tip2019/index.html

Telefónica executive’s comment
Endeavor to Develop Open and Intelligent
Cloud-native Transport NetworksÓscar González de Dios
Telefónica I+D global CTO
In the past years, Telefónica has been upgrading
its IP and optical transport networks in a series oftransformation projects, internally named“FUSION”, in order to deliver the transport
requirements of the customers. However, theapproach on fully relying on vendors’ proprietary
solutions has led to delays in introducing new
functions and innovations. Moreover the interfaces among the Operational support systems
have been proprietary, non-programmable andclosed interfaces that prevent new applications
from a rapid and automated introduction.
Telefónica is currently working on the full definition of Software Defined Transport Networkmodel, named “iFUSION”, as main pillar of the
network transformations, which covers the SDNbased control for core, metro and backhaul network segments including IP/MPLS, optics andmicrowave technologies. In order to introduce the
network softwarization principles effectively,
Telefónica chooses the standard technologies,
especially those that are more mature in the

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

industry, for interfaces andservice models.
On the other hand, while
the fully aggregated modelin optical networks is perfectly valid regarding its
operation, it has prevented
from quickly incorporating
new transmission technologies from other vendors.
In packet transport, hardware and software are
currently coupled, so the capacity, features, andoperation of the routers are tightly coupled. Inthat sense, Telefónica has an internal program,“openFUSION”, to foster the partial disaggregation of optical networks and commoditizing IP
routers into white boxes.
Telefónica has joined NTT to lead the CANDI
initiative in an effort to foster openness in the
Telco industry. The CANDI initiative in which
NTT and Telefónica collaborate together withother operators is the perfect place to develop thesolutions for an operator open network and drive
the industry. With the PoCs, the CANDI initiative
will continue to demonstrate the telco use caseswith open standard interfaces, coordinated IP/
Optical control and disaggregation.

81

Short Reports

NTT researcher’s comment
Towards Flexible and Reliable Networks with
Open Technologies
Minoru Yamaguchi
Transport Network Innovation Project,
NTT Network Service Systems Laboratories
NTT Network Service Systems Laboratories is
researching and developing MSF and Beluganos
and discloses them to achieve networks in which
free choice and operation that meet operator
requirements are possible as with server virtualization technology. These activities promote ahighly flexible and reliable network. NTT’s
activities received approval from TIP, and a subgroup in which operators discuss use cases andarchitectures was established to drive disaggregation and open innovation. This was the first time
we managed a community and conducted a PoC
with carry-in equipment outside Japan. We conducted technical discussions with five operatorsand ran the secretariat inparallel. We also constructed an environment to conduct this PoC. We went toTelefónica Lab in Spainand built the environment
in cooperation with Óscar
and other participants. The
PoC sometimes did not
p r o g r e s s a s ex p e c t e dbecause of time differences and language barriers
as well as the circumstances of each company, but
we could share common requirements and issues
by organizing discussion items and roadmaps. We
can interest many operators, vendors, and system
integrators by spreading the results of this PoC as
common requirements of operators, and we feltthat our activities have great appeal. We would
like to do our best for achieving flexible and reliable networks with open technologies by involving many people and working together.

For inquiries:
Public Relations Office, Planning Department,
NTT Information Network Laboratory Grouphttps://www.ntt.co.jp/news2019/1910e/191016b.
html

NTT Technical Review

Vol. 18 No. 3 Mar. 202082

External Awards
SG13 Fellow
Winner: Yoshinori Goto, NTT Network Technology Laboratories
Date: October 25, 2019
Organization: International Telecommunication Union - Telecommunication Standardization Sector (ITU-T) Study Group (SG) 13
Mr. Goto has served ITU-T SG13 for many years. He has been vicechairman of SG13 since he was elected at the World Telecommunication Standardization Assembly (WTSA) in 2012. Through his
extraordinary commitment and expertise, he has been successfully
managing Working Party 2 of SG13, which includes all activities of
the SG related to cloud computing and big data. Furthermore, he hassignificantly supported the SG13 chairman in managing the SG byleading important ad hoc groups related to the preparation of SG13
for the next study period as well as in creating ITU-T Focus Groups
under the partnership of SG13. He demonstrated his high managerial
and negotiation skills when leading activities in SG13.
Certificate of Service
Winner: Seishi Takamura, NTT Media Intelligence Laboratories
Date: November 14, 2019
Organization: Picture Coding Symposium
For his contributions as an organizing committee member and session chair.
Network Slicing Implementation 2019 Award
Winner: Takuya Tojo, Hiroki Baba, Shiku Hirai, NTT Network
Technology Laboratories; Soushirou Abe, Mitsuo Amasaka, Tomohiro Okada, Kazuto Toyozumi, Takayuki Nakamura, Masaya Nakouji, Aki Fukuda, Masashi Funada, Akira Matsuda, NTT Network Service Systems Laboratories, Yuuki Sakaue, Tatsuya Fukui, NTT
Access Network Service Systems Laboratories
Date: November 21, 2019
Organization: MEF (Metro Ethernet Forum)
For “5G xHaul Sharing as Slices with LSO Orchestration.”
Best Paper
Winner: Xiaoxi Zhang, Machiko Shinozuka, Yuriko Tanaka, NTT
Network Technology Laboratories; Yuko Kanamori, Toshihiko
Masui, National Institute for Environmental Studies
Date: November 27, 2019
Organization: The 11th International Symposium on Environmentally Conscious Design and Inverse Manufacturing (EcoDesign
2019)
For “Forecast of Future Impacts of Using ICT Services on GHG
Emissions Reduction and GDP Growth in Japan.”Published as: X. Zhang, M. Shinozuka, Y. Tanaka, Y. Kanamori, andT. Masui, “Forecast of Future Impacts of Using ICT Services on
GHG Emissions Reduction and GDP Growth in Japan,” Proc. of
EcoDesign 2019, pp. 88–95, Yokohama, Japan, Nov. 2019.
Best Paper Runner Up
Winner: Seishi Takamura, Media Intelligence Laboratories; Atsushi

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Shimizu, NTT TechnoCross Corporation
Date: November 27, 2019
Organization: 2019 IEEE International Conference on Visual Communications and Image Processing (VCIP 2019)
For “Water-bottom Video Coding Based on Coding-oriented Reference Frame Generation.”
Published as: S. Takamura and A. Shimizu, “Water-bottom Video
Coding Based on Coding-oriented Reference Frame Generation,”
Proc. of VCIP 2019, Sydney, Australia, Dec. 2019.
Best Research Paper Award
Winner: Ryo Ishizuka, Hironori Washizaki, Yoshiaki Fukazawa,
Waseda University; Shinobu Saito, Saori Ouji, NTT Software Innovation Center
Date: December 14, 2019
Organization: The 10th International Workshop on Empirical Software Engineering in Practice (IWESEP 2019)
For “Categorizing and Visualizing Issue Tickets to Better Understand the Features Implemented in Existing Software Systems.”
Published as: R. Ishizuka, H. Washizaki, Y. Fukazawa, S. Saito, andS. Ouji, “Categorizing and Visualizing Issue Tickets to Better Understand the Features Implemented in Existing Software Systems,” Proc.
of IWESEP 2019, Tokyo, Japan, Dec. 2019.
2019 IEICE Communications Society OCS Young
Researchers Award
Winner: Hiroki Taniguchi, NTT Network Innovation Laboratories
Date: December 17, 2019
Organization: The Institute of Electronics, Information and Communication Engineers (IEICE) Communications Society TechnicalCommittee on Optical Communication Systems (OCS)
For “255-Gb/s PAM-8 O-band Transmission Using MLSE Basedon Nonlinear Channel Estimation with 20-GHz Bandwidth Limitation.”
Published as: H. Taniguchi, S. Yamamoto, M. Nakamura, and Y.
Kisaka, “255-Gb/s PAM-8 O-band Transmission Using MLSE Basedon Nonlinear Channel Estimation with 20-GHz Bandwidth Limitation,” IEICE Tech. Rep., Vol. 119, No. 93, OCS2019-18, pp. 43–46,
June 2019.
IDW ’19 Best Paper Award
Winner: Munekazu Date, Shinya Shimizu, Hideaki Kimata, NTT
Media Intelligence Laboratories
Date: December 24, 2019
Organization: The 26th International Display Workshops (IDW ’19)
For “Depth Range Control in Visually Equivalent Light Field 3D(VELF3D).”
Published as: M. Date, S. Shimizu, and H. Kimata, “Depth Range
Control in Visually Equivalent Light Field 3D (VELF3D),” Proc. of
IDW ’19, Vol. 26, pp. 65–68, 3DSA3/3D3-1, Sapporo, Japan, Dec.
2019.

83

Papers Published in Technical Journals andConference Proceedings
Maximum Entropy Method without False Peaks with Exact
Numerical Equation
F. Ishiyama
Journal of Physics: Conference Series, Vol. 1438 (Proc. of CCISP
2019, Phuket, Thailand, Nov. 2019), 012031, January 2020.
The standard numerical maximum entropy method (MEM) stilluses the Yule-Walker equation, which contains rough approximation
by Walker. The commonly used numerical equation contains additional modifications to reduce calculation cost. We now have powerful computers, so there is no reason to use this modified equation. We
argue that the drawbacks of MEM, such as false peaks and peak splitting, are from these modifications. They do not appear when using theexact numerical equation, even a given time series is fractional.

NTT Technical Review

Vol. 18 No. 3 Mar. 2020

Verifying Commuting Quantum Computations via Fidelity
Estimation of Weighted Graph States
M. Hayashi and Y. Takeuchi
Proc. of Quantum Information Processing Conference 2020, Shenzhen, China, January 2020.
We proposed methods for verifying whether a weighted graphstate, which is a useful resource state for quantum computing, can befaithfully prepared. We also applied our methods to verify quantum
supremacy demonstrations.

84