NTT Technical Review

March 2021 Vol. 19 No. 3
View from the TopNaoki Tani, Executive Vice President, Chief Technology Officer, Executive General Manager
of R&D Innovation Division, NTT DOCOMO

Front-line Researchers

Koji Muraki, Senior Distinguished Researcher, NTT Basic Research Laboratories

Feature Articles: NTT R&D Forum 2020 Connect Special Sessions
Sports & Live Entertainment Viewing Re-imagined in the Post-coronavirus Era

Challenges of NTT Space Environment and Energy Laboratories in the Coming Space
Millennium

Feature Articles: Media Robotics as the Boundary Connecting Real Space and Cyberspace

Toward Cyber-physical Interaction for Natural Connection of Real Space and CyberspaceImproving Depth-map Accuracy by Integrating Depth Estimation with Image SegmentationAffect-perception Control for Enhancing a Sense of Togetherness for Remote Spectators

Visible-light Planar Lightwave Circuit Technology and Integrated Laser-light-source Module
for Smart Glasses

Fine-grained Hand-posture Recognition for Natural User-interface Technologies



Information-display Method for Reducing Annoyance by Gaze Guidance

Presenting Material Properties with Mid-air Pseudo-haptics



Evaluation of Adaptability to Unfamiliar Environments Using Virtual Reality

Regular ArticlesIncreasing Capacity of a Strictly Non-blocking Clos Network Composed of Optical SwitchesGlobal Standardization Activities


Recent Standardization Activities in ITU-T on Single-mode Optical Fiber and Space Division
Multiplexing Technologies

Short Reports


Simulating Complex Quantum Networks with Time Crystals

World’s Fastest Directly Modulated Laser Exceeding 100-GHz BandwidthDemonstration of a Novel Method to Generate Chaotic Signals Using a MEMS Oscillator

External Awards/Papers Published in Technical Journals and Conference Proceedings

View from the Top

Let’s Change, Challenge, andImplement for Creating a “Wellbeing
Society”
Naoki Tani
Executive Vice President, Chief Technology
Officer, Executive General Manager of R&D
Innovation Division, NTT DOCOMO
Overview

NTT DOCOMO became a wholly owned subsidiary of NTT at the end of
2020. With the launch of 5G (5th-generation mobile communication system) services, its business environment is undergoing a period of majorchange. Society is also changing due to the novel-coronavirus pandemic,
such as acceleration of digital transformation and spread of remote working. We asked Naoki Tani, executive vice president, chief technology officer, executive general manager of R&D Innovation Division of NTT
DOCOMO, about the company’s efforts to address social issues using
technology and how he communicates with employees in this period ofchange.
Keywords: wellbeing, cyber-physical fusion, Mobile Spatial Statistics

Technology and R&D will lead to creation of ahappier future—First, tell us about the environment surrounding
NTT DOCOMO.
We can talk about the environment surrounding
NTT DOCOMO from two perspectives: the businessenvironment surrounding NTT DOCOMO and thatsurrounding the NTT Group. As for the first perspective, in 2020, NTT DOCOMO and other communication companies began providing 5th-generation
mobile communication system (5G) services, and thecompetitive environment changed accordingly. As
the second perspective, on December 29, 2020, NTT
DOCOMO became a wholly owned subsidiary ofNTT.
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

5G services do not simply provide new technologies to society, they also aim to address social issues
and create new value. For this reason, in February
2018, NTT DOCOMO initiated the DOCOMO 5G
Open Partner Program, which provides partner companies and organizations (referred as “partners”) withinformation on 5G technologies and specifications aswell as a place for exchanging ideas among them. In
addition, we established the DOCOMO 5G OpenLab, where partners developing 5G services can test
their services with 5G equipment such as 5G radio
stations.
In September 2019, we started our 5G pre-commercial services to enable us and partners to create business using the same network equipment and frequency bands as 5G commercial services and enable users
to experience the same environment as 5G commercial
1View from the Top

services. In March 2020, we launched our fullyfledged 5G commercial services.
Different companies have begun providing 5G services and are promoting 5G’s key features, namely,
high speed and large capacity. NTT DOCOMO isfocusing on solving social issues and creating newvalue with 5G as well as offering attractive billing
plans for users. We have also been leading the international standardization of 5G together with interested parties around the world, and we want toactively promote international standardization
beyond 5G, i.e., Beyond 5G and 6G.
NTT DOCOMO has become a wholly owned subsidiary of NTT, so we want to put in place a thoroughsystem so that the research and development (R&D)
activities of both companies will be more closely
integrated than ever before. NTT DOCOMO is anoperating company: we operate in an environment inwhich customer needs can be directly reflected in
R&D and the R&D results can be directly fed back tobusinesses. Strengthening R&D cooperation between
NTT and NTT DOCOMO makes it possible to widen
the communication pipeline between the R&D and
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

business departments. In this manner, we want tomake the cooperation more coordinated.
I believe that the purpose of NTT DOCOMO is topromote communication between people. To accelerate our R&D, we have set up the “wellbeing society”
as a vision for the future, and we are engaged in R&D
that will contribute to building a people-friendly society where individual needs can be carefully addressed.
In other words, our efforts to address social issues
will lead to the creation of a society in which people
can live happy, enjoyable lives. Our approach is tobuild an ecosystem based on three perspectives:
industry, society, and individuals. We will develop
industries that benefit people, create a social environment in which people can live comfortably, and support people in demonstrating their abilities and live
enjoyable and happy lives. I believe that it is our mission to think about what kind of R&D is necessary tomeet these goals, and we want to share these goals in
our R&D collaboration with NTT.

2

View from the Top—Could you give us an example of R&D for creatinga “wellbeing society”?
The current focus of our R&D on technologies andservices that will create such society is a framework
called cyber-physical fusion. This framework will
also contribute to actualizing the Innovative Opticaland Wireless Network (IOWN) being advocated by
the NTT Group. People, things, and events in the realworld (physical space) are digitized, their data are
acquired and accumulated in cyberspace, those data
are analyzed to predict the future, and the acquired
knowledge is fed back (actuating) to the real world.
By repeating the above-mentioned process as a loop,
it is possible to create value for people, industries,and societies in the real world, such as providing newexperiences, improving efficiency and productivity,
facilitating optimization, and ensuring safety andsecurity.
NTT DOCOMO is also taking up the challenge of
evolving core technologies to enable cyber-physicalfusion. These core technologies include artificial
intelligence, which predicts the future and generates
knowledge by linking various acquired and accumulated data, Internet of Things (IoT) and device technologies, which provide a means of digitizing the realworld and knowledge feedback, and a network thatconnects the real world and cyberspace. We want tocreate a “wellbeing society” by evolving these coretechnologies and establish a platform that can beshared across industries.
As the spread of the novel coronavirus generates
restrictions on real-world activities, the shift to thevirtual world and online communication is a continuing trend. Following this trend, we hosted “docomo
Open House 2021,” held from February 4 to 7, 2021,
in a completely online format and provided virtualbooths that attendees could visit and enjoy the eventand exhibitions. For example, we set up a boothwhere attendees could experience traveling abroad
and another in which they could enjoy sports andmusic with free-viewpoint videos.
Sometimes you may want to see more of or actually
see with your own eyes things that you experienced
virtually. I think it’s important to bring out this feeling by providing a virtual world. The fact that the realworld comes after the virtual world means that thegreater the value of being able to experience something virtually, the greater the value of it in the realworld. I sometimes hear people say that real-world
experiences will be replaced with virtual ones, but I
beg to differ; there will be more cases in which realsocial activities are invigorated through virtual ones.
All employees have the potential to unlock
the future—Expectations concerning efforts to improve the realworld through the fusion of the real and virtualworlds are rising.
Is the reader aware of the Mobile Spatial Statistics?
Among the news about the novel coronavirus, the rateof increase or decrease in the number of people atmajor train stations has been reported in Japan, andstatistical information calculated using our Mobile
Spatial Statistics is also used. The Mobile Spatial
Statistics is a population statistics service createdusing the mechanisms of NTT DOCOMO’s mobile
phone network and provides population-distribution
data by gender, age group, and place of residence inreal-time minus a minimum of one hour in 10-minuteincrements [1]. We also provide a service that visualizes

NTT Technical Review

Vol. 19 No. 3 Mar. 20213

View from the Top

the population of Japan by using high-speed visualization technology developed by NTT laboratories.
Using this service, with search/read speed that isseven-times faster than previously possible, users cansmoothly display and browse changes in the population of Japan over time. Gender, age, and place ofresidence in relation to each square (500 m2) of amesh covering the whole country can be freely selected and displayed.
As an actual example, a forecast model created in
combination with data from a partner is used by store
owners to understand the population increase or
decrease on a certain day in a particular commercial
area. As a result, they can predict sales and adjust the
production of lunch boxes and delicatessen items to
reduce food loss. Moreover, road-management operators can predict the occurrence of traffic jams on
managed roads by determining the population
increase or decrease on a certain day in a specific areasuch as a sightseeing spot. That highly accurate traffic-jam-prediction information is also provided todrivers through smartphone apps, etc., helping toeliminate traffic jams. These cases are examples of
embodying cyber-physical fusion and creating services by using the foresight in identifying value inaccumulated information, planning ability, and analytical ability.
Although I have talked about services originatingfrom the R&D Department, NTT DOCOMO has amechanism in which various ideas of employees canbe used in new businesses, and the R&D Department
supports this mechanism. We will continue to evaluate the possibility of practical application and commercialization of ideas proposed by employees andmake them a reality. In this way, I want to value thepossibilities possessed by employees other than those
directly involved in R&D and services. I believe their
possibilities are waiting to be unlocked.
—Can you tell us what you have cherished as a top
manager thus far?
The social environment has been changing drastically due to the novel-coronavirus pandemic. I think
this change gives us an opportunity to continually
come up with new ideas. In such a rapidly changing
environment, we should consider three important
points: “change,” “challenge,” and “implement.”
These may be words you often hear, but I have thefollowing viewpoints.
Let’s start with “change.” Take a positive mindset
that every challenge is an opportunity for change.
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

“Challenge” is to create a story through your own
will. It’s important to draw the big picture and act.
“Implement” means start quickly with what you cando. It doesn’t matter if you “start small.” The accumulation of experience little by little will produce outstanding results.
Since many people at NTT DOCOMO are engaged
in R&D, the directions of their work may differ. If
that is the case, it is important to return to the starting
point. During discussions with employees about
work or a project, I often ask, “What are we doing thisfor?” because I sometimes feel that it is necessary to
discuss their work from a broader perspective. At
such times, I try to improve communication so that
we can share the starting point.
It’s very important to listen to employees. Rather
than telling them to do something, I always want tohear what they think and feel. When I find it difficult
to understand what a person is saying, by listening
patiently, I can understand his/her true intention.
Even if the discussion becomes complicated,
responding candidly with “Hold on. What are wedoing this for?” will calm the situation and allow us
to return to the starting point, share it, and move forward together. Listening to employees also leads tounderstanding the diversity and discovering thepotential of employees.
Return to the starting point to understand thestory and purpose—As your reputation suggests, you seem to be aneasy-to-talk-to boss.
Although that may be true, sometimes employees
are reluctant to talk to me in my position as their boss,
so I try to talk to them as much as possible. Since it is4View from the Top

difficult to take time for each employee, I try to talkto them during meetings. Also, some employees
come to see me by spotting gaps between meetings,
and I value that kind of time as much as possible.
Since we are teleworking these days, I sometimes feel
frustrated because I can’t communicate with employees easily. Regardless, I still try to have the opportunity to speak to people as much as possible.
As I said above, while listening to everyone, I haveviewed everything positively to allow us to “change,”“challenge,” and “implement.” Prioritization is the
key to execute these three points. That is to say, selectmatters so that you can focus on what you think isimportant. For example, when creating materials, it isimportant that they are written so that the purpose of
the materials can be understood easily. The next
important point is where and to whom to talk and how
much time to spend. Bosses and customers—as well
as external lectures and management meetings—arecompletely different. What you can convey in 30 seconds differs from what you can convey in one hour. It
is very important to devise a way of communicatingthat firmly corresponds to these different situations.
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Figures, design, wording, etc. all need to be applied
according to the purpose. Therefore, I often ask the
person who created such material a question, “Whatis this used for?” to remind us of the starting point andallow us to understand its story and purpose. I’m
always aware of this question, and I want all employees to be aware of it too.
—As a chief technology officer, please give a few
words to researchers/engineers.
After joining the company, I worked in the R&D
Department of NTT DOCOMO for about 20 years.
After working in the Network Department of the
Kansai Branch and the IoT Business Department, I
returned to the R&D Innovation Division after being
away for about nine years. Technology has evolved
tremendously over those last nine years.
In the IoT Business Department, where I worked
until June 2020, I was working with the aim of creating new business models and increasing profit withpartners by using new technologies. During that
period, I was observing NTT DOCOMO’s R&D from5View from the Top

the outside. After assuming the post of chief technology officer last year, I reaffirmed that NTT DOCOMO
possesses many outstanding technologies, and thesetechnologies can be used in our business more thanever.
I encourage our researchers/engineers to take pridein our own technologies and firmly understand themeaning, significance, and position of those technologies. If you keep doing this, you will find opportunities. There are many situations in which thetechnologies we possess can help solve social issues.
If you find an opportunity, you can only take it. I’dlike you to keep in mind the three key points I focusedon, namely, “change,” “challenge,” and “implement.”

Interviewee profile
Career highlights
Naoki Tani joined NTT in 1989 and worked in
the R&D Department of NTT DOCOMO from1992 to 2011. He served as head of the Network
Department of Kansai Branch from 2011 to 2014,
head of the M2M Business Department from2014 to 2015, head of the IoT Business Department
from 2015 to 2020, and executive officer of NTT
DOCOMO from 2017 to 2020. He has been in his
current position since June 2020.

Reference[1]

Mobile Spatial Statistics (in Japanese), https://www.nttdocomo.co.jp/
biz/service/spatial_statistics/

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

6

Front-line Researchers

Creating a Map of Science andConnecting with the World andFuture through It
Koji Muraki
Senior Distinguished Researcher,
NTT Basic Research Laboratories
Abstract
Researchers at the Quantum Solid State Physics ResearchGroup of NTT Basic Research Laboratories focus on the manybody and correlation effects caused by the interaction between
electrons as well as on quantum-mechanical properties of electrons such as wave nature, superposition state, and spin. By engineering and controlling these properties of electrons using theheterostructure and nanostructure of semiconductors and atomiclayer materials, the group is investigating quantum devices andsensing techniques that cannot be obtained with individual electrons. Such research is expected to enable the development of
ultralow-power-consumption devices and highly sensitive sensors. We asked Koji Muraki, a senior distinguished researcher and leader of this group about the progress of his research and the attituderesearchers should have.
Keywords: topological insulator, quasiparticle, semiconductor heterostructure

Quantum correlation brings out the potentialof electrons—Please tell us about your current research.
My research aims to obtain physical properties notfound in conventional semiconductors and new functions exploiting such properties by using (i) a structure in which different types of semiconductors are
artificially laminated (i.e., heterostructure) or (ii) a
microfabricated structure (i.e., nanostructure). In
particular, I’m focusing on the quantum-mechanicalproperties of electrons, especially a property called
spin, i.e., having magnetic properties of small magNTT Technical Review

Vol. 19 No. 3 Mar. 2021

nets, and the many-body effect, which is caused by alarge number of electrons acting as a group rather
than moving around independently.
We are pursuing such research of physical phenomena to engineer and use the properties of electrons
that are not used in conventional devices to enable thedevelopment of ultralow-power-consumption devices
and highly sensitive sensors. Our ultimate goal is tocreate a quantum computer based on a new operating
principle that overcomes the weakness of a conventional quantum computer, i.e., the quantum statechanges due to noise (i.e., low error tolerance) (Fig. 1).
We are taking two broad approaches to thisresearch. One approach uses the many-body effect7

Front-line Researchers

Quantum state composed of two quasiparticles
Superposition
state (quantum)
1
0
or

e−

1
0

Noise

e−

Noise

0
1
1
0

0

Quasiparticle

1
Classical bit

Quantum bit (Qubit)

Quasiparticle

Qubit using quasiparticles

Quantum state changes
due to noise

Quantum state does not change
due to noise (error tolerance)

Fig. 1. Quantum bit using quasiparticles.

that I mentioned above, and the other relates to spin
and uses a new kind of material called a topologicalinsulator.
Before explaining the approach using the manybody effect, I need to explain the so-called quasiparticle. For an example of a quasiparticle, let’s take alook at a p-type semiconductor, which is a transistor
component. If electrons that are negatively charged
are missing in the semiconductor’s energy levels thatthey are supposed to fill, those states, called holes(because they look like holes from which the electrons have escaped), behave like positively charged
particles. Such holes have essentially the same properties as electrons. Our goal is to create quasiparticles
with properties that are essentially different fromelectrons by exploiting the many-body effect.
The other approach is to use a topological insulator.
The concept of topological insulator was first proposed in 2005 and is a very new concept in the history of physics. The 2016 Nobel Prize in Physics wasawarded to the three physicists who created the theory underlying topological insulators. An insulator is amaterial that does not conduct electricity. According
to the theory proposed in 2005, there are two types of
insulators: topological insulators and ordinary insulators, and a layer that conducts electricity always
exists at the boundary between the two types of insulators. In 2013, our group experimentally demonstrated that a topological insulator can be formed
using a heterostructure of semiconductors [1] (Fig. 2).
Although various materials have been reported as
topological insulators, the topological insulator creNTT Technical Review

Vol. 19 No. 3 Mar. 2021

ated using our semiconductor heterostructure can beelectrically controlled and switched between a normal insulator and topological insulator by applying a
voltage to the gate electrode. In fact, we were thesecond team in the world to experimentally show that
this material system (i.e., the semiconductor heterostructure shown in Fig. 2) can form a topologicalinsulator. The first team published a paper showingthat conduction occurs at the edge of the system, andwe took time to expand on that paper and showed thatthe system is a topological insulator, that is, it does
not conduct electricity excluding the edge of the system. Our paper is still often cited, but being number
two made me think about how to present our results.
—You were interviewed eight years ago, and you’ve
achieved numerous results since then.
That interview [2] eight years ago was just after the
results of an experiment using a method called nuclear magnetic resonance to show that the spin of electrons under special states are aligned in the same
direction due to the many-body effect. The resulting
paper was published in Science in 2012 [3]. Since thedirection of the spin determines whether the electron
can occupy the same position, the paper was highly
evaluated as indicating that the electron is in the special state predicted by theory. In 2014, using nuclear
magnetic resonance once again, we demonstrated
that under different conditions, the electrons form alattice like a crystal and become immobile, and that
paper was published in Nature Physics [4]. Although8Front-line Researchers

Number of holes: 0

Number of holes: 1
Topology: properties (suchas number of holes) thatcharacterize the shape ofobjects that do not changeeven when deformed

Ordinary insulator

Topological insulator

Inverted
bands

EnergyConduction
band
Valenceband

Formation of a topologicalinsulator using a semiconductor
heterostructure

AlGaSblnAs
GaSb

Insulating
regionInsulating
region

Does not conduct electricity

Current flows along theedge of the sample

AlGaSb
50 nm

Electrons with up spin

Electrons with down spin

The quantum state is maintained because the direction of electron motion and that of spin are opposite each other.
The quantum-mechanical properties of the topological insulator are stable just like the number of holes formed by the handle
and body of a coffee cup does not change by a continuous deformation.
AI: aluminum
InAs: indium arsenide
GaSb: gallium antimonide

Fig. 2. Formation of a topological insulator using a semiconductor heterostructure.

this paper was published because it was seen as newexperimental evidence supporting the long-held phenomenon known as electron crystallization, it was
not fully satisfactory for us in regard to the original
purpose of clarifying the properties of quasiparticles.
Up until that time, I had been investigating thenature of electrons that form the background, or
matrix, of quasiparticles; however, more-precise
measurement techniques for investigating the properties of quasiparticles have become necessary. Therefore, I invited a researcher at a university to join mygroup who was interested in the same topic and had
the necessary skills and knowledge. Until then, my
research style had been to carry out every procedureby myself from sample preparation, measurement,
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

and theoretical analysis; however, I thought I
shouldn’t keep doing that. For example, the charge on
a quasiparticle is smaller than the charge on an electron, and the number of research groups in the worldthat can measure it is limited. As reported in a recently published paper, by combining nuclear magnetic
resonance and quasiparticle-charge measurement, wewere able to clarify how the electronic state changes
when an electric current is applied. That idea camefrom a member of my group, not myself.
The next step after forming a topological insulator
with a semiconductor heterostructure is to control it
electrically; however, this step also faces a problem.
That is, an insulator was formed only under very limited conditions, and that limitation was inconvenient9Front-line Researchers

in regard to conducting various experiments. Therefore, by intentionally incorporating materials with thedifferent lattice constant and using the strain in a
crystal, in 2016, we were able to improve the properties as the insulator, and in 2020, it became possible
to switch between topological insulators and ordinary
insulators electrically while maintaining the insulator
properties. These accomplishments required a steady
accumulation of experiments, such as crystal growthand improvement of the gate insulating film, andthose who worked hardest were the young researchers in the group. Although it is now known that various materials are topological insulators, no one hasachieved the expected results with regard to the spin
properties of the edge of an insulator, and we believethat a real breakthrough will be achieved in the future.
Researchers have no apprenticeship period—In regard to your research activities, what do you
feel has changed since the last interview?
At that time, I was working with a small team consisting of a postdoctoral fellow (a researcher in alimited-term position after obtaining a doctoral
degree) and two or three other researchers; however,the number of members involved in the project has
now increased. It’s my responsibility to coordinate
shifts so that young researchers can conduct experiments while taking measures to prevent the spread of
the novel coronavirus and support them so that they
can devote themselves to research. It is the younger
members of the group who are directly involved inchallenging experiments.
As a result of the change in my position, I sometimes feel it frustrating that I have less chance todirectly engage in experiments. For example, if
you’re experimenting by yourself, you’re the first to
see the results; in contrast, when you are working ina group, young researchers conduct the experiment,so they will see the experimental results first. By thetime I get to know the research results, a process bywhich the group members notice something about theresults and report it to me had been added. In otherwords, the results I received had been filtered byother researchers. However, that change is not such abad thing because other group members can noticethings that I might not notice, and the group canachieve results that cannot be achieved with just one
person. In consideration of the fact that the members
bring proposals and ideas that cannot be thought of by
oneself, their contribution is overwhelmingly greater
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

than any amount of my frustration. I feel that I am
working in a very blessed environment.
In a position to manage the group, I think it isimportant to make use of the abilities of individual
researchers with multiple interests and specialties sothat they can move in the right direction both individually and as a group. Since researchers will not be
enthusiastic about a subject they are not interested in,
I think it is essential to create an environment inwhich each member of the team can do what theywant to do. For example, even if you can produce
results when doing something other than what you
want to do, you won’t be able to continue producing
results in that way. I think unless you feel that you are
the one who produced the result or you think nobody
else could have produced the result, you won’t get atrue sense of accomplishment.
What’s more, I think that many people who produce
results are good at building relationships based ontrust with those around them and can collaborate withthem. Just because you think about managing a groupdoes not mean that the group will produce goodresults. However, we are currently able to come up
with ideas without hesitation under very good relationships based on trust, so I believe we can continue
to produce good results.
—Has there been any change other than research
activities during the stay-at-home request period toprevent the spread of novel coronavirus infection?
My passion for the electric guitar, which was myhobby when I was a student, has rekindled. I hadn’t
played it for a long time, but I revived it while I wasstaying at home after the state of emergency was
declared in April 2020. The electric guitar does not
feel right to play unless it makes a loud sound froman amplifier. One reason for my renewed interests isthat thanks to digital technology, I can feel as if I’m
playing at a loud volume in a concert hall just by
generating sound in my headphones. Moreover, manychannels on an Internet streaming site host my favorite guitarists explaining how to play, and others allow
amateurs to show off their performances. The feelingof being able to connect with people around the world
through being interested in the same things is similar
to research.
However, research has a special aspect. In the caseof music, everyone starts out as an amateur, and only
a handful of them become professionals. In contrast,
as soon as a researcher becomes a member of a laboratory, they will be treated as a full-fledged researcher
10Front-line Researchers

and required to present their research results in papers
and at academic conferences. If you have a job inwhich you also write articles, like a newspaper
reporter, I imagine that no matter how many articles
you write, they may be turned down by the chief editor and not published in the paper. However, if you
are a researcher, you will be required to publish yourresearch results in the form of papers, unless you turnthem down yourself. As I said in the last interview,
researchers suddenly become professionals without
an apprenticeship period or rehearsals.
What I realized this time is the research environment of researchers differs depending on the institution to which they belong, such as companies anduniversities. For example, researchers at universities
will spend more time acquiring research funding thancorporate researchers. As a researcher at a company,
I realized that I was fortunate to spend less timeacquiring research funds than researchers at universities. I thought that that’s why I had to discipline
myself and put more effort into my research. Sometimes it is important to look at yourself objectively
like that and confirm the value of your existence.
I want to achieve results for which it can be said
“That was a breakthrough.”—Confirming the value of existence has a philosophical sound. How do you see the work of a researcher,Dr. Muraki?
I think that researchers are self-fulfilling throughtheir research and have the joy of connecting with theworld and strangers through research. Throughresearch, we create a map of science and connect withthe world and future through that map. If I recall correctly, I also mentioned in my last interview that thework of a basic researcher is to make a map and thatmaking a map is an adventure; I haven’t changed thatway of thinking. Unless your research results wereincorrect, you should be able to trust the map you
made. Even if the map (research) you are working on
is not directly useful, it will be useful in some way. I
think that that map can be left to posterity.
However, the way we pursue adventure has changed
a lot from eight years ago. Currently, our juniorresearchers are actually conducting experiments, and
I’m in a position to ask questions and give advice
based on their results. Although we’re going throughthis adventure together, it’s clear that I’m actuallyless likely to see the results with my own eyes ortouch them with my own hands. On the contrary, it is
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

fun to learn from junior researchers, and I’m happy
that such opportunities have increased. I want to workwith young people doing work for which we can say,“That was a breakthrough.”—Please give a word of advice to the next generation.
When talking to young researchers at academic
conferences, it can be overwhelming to find people
who are familiar with not only the current state-ofthe-art research but also the research done before they
were born. I think it’s a privilege for young people tobe able to study thoroughly what they are interested
in. Of course, I’m researching what I’m interested intoo, but how much time and energy I can devote to
one research topic is far from what young researchers
can do.
I think that most young researchers, with a few
exceptions, always have a feeling that they can’t dowhat they want to do, don’t produce good results,
don’t seem to be recognized, or feel gloomy. However, those feelings always accompany the privilegeof youth, so I want you to concentrate on making themost of that privilege. It would be unworthy of you toworry too much and do not do what you want to do.
That is why I want you to broaden your options andcultivate negotiation skills to open up your path.
I recently noticed that there are cases in whichyounger generations who used to be trainees or postdocs at our laboratories have achieved great results
after they had their own research groups and various
careers. I think that although their careers as researchers were not plain sailing, the fact that they have diligently accumulated results and built a solid platformover a long period has paid off. Looking at thesepeople, I once again feel that the life of a researcher
is pretty long. What they all have in common is anattitude of not giving up.
To live as a researcher, it is important to have thedepth to delve into what you are interested in and thewidth to observe broader areas. I also want you totake an exit-oriented approach striving for the results
you aim for in addition to an entry-oriented approach
focusing on research themes. Many basic researchers
may be entry-oriented, but you should continue to digfrom not only one side of the entrance but also fromthe exit while exploring various possibilities. The
idea of digging at various places from both sides of
the tunnel may be effective in the long run. Therefore,
let’s listen to the opinions of a wide variety of people
to expand our possibilities.

11Front-line Researchers

References
[1]
[2]

[3]
[4]

K. Suzuki, Y. Harada, K. Onomitsu, and K. Muraki, “Edge Channel
Transport in the InAs/GaSb Topological Insulating Phase,” Phys. Rev.
B, Vol. 87, No. 23, 235311, 2013.
Front-line Researchers, Koji Muraki, NTT Technical Review, Vol. 10,
No. 10, 2012.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201210fr1.html
L. Tiemann, G. Gamez, N. Kumada, and K. Muraki, “Unraveling theSpin Polarization of the ν = 5/2 Fractional Quantum Hall State,” Science, Vol. 335, No. 6070, pp. 828–831, 2012.
L. Tiemann, T. D. Rhone, N. Shibata, and K. Muraki, “NMR Profilingof Quantum Electron Solids in High Magnetic Fields,” Nat. Physics,
Vol. 10, pp. 648–652, 2014.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Interviewee profile
Koji Muraki
Senior Distinguished Researcher and Group
Leader of the Quantum Solid State Physics
Research Group, NTT Basic ResearchLaboratories.
He received a B.E., M.E., and Ph.D. in applied
physics from the University of Tokyo in 1989,
1991, and 1994. He joined NTT Basic ResearchLaboratories in 1994. From 2001 to 2002, he wasa visiting researcher at the Max Planck Institutefor Solid State Research, Stuttgart, Germany. His
research interests are focused on the many-bodyeffect in low-dimensional semiconductor structures. He is a member of the Physical Society ofJapan and the Japan Society of Applied Physics.

12

Feature Articles: NTT R&D Forum 2020 Connect
Special Sessions

Sports & Live Entertainment Viewing
Re-imagined in the Post-coronavirus
Era
Shingo Kinoshita
Abstract

On November 19th, 2020, special session 1 of NTT R&D Forum 2020
Connect was held and live streamed. The session welcomed guests Yuki
Ota, president of the Japan Fencing Federation and vice president of theInternational Fencing Federation and Haruyuki Moroishi, CEO and CCO of
IMAGICA EEX Inc. and general producer of IMAGICA GROUP. The
author, Shingo Kinoshita, executive research engineer, NTT Service Evolution Laboratories, moderated a discussion on the theme “Sports & LiveEntertainment Viewing Re-imagined in the Post-corona Era.” During thesession, they discussed the current status and future of sports and live entertainment during the novel coronavirus pandemic.
Keywords: sports, live entertainment, experience-oriented

1. Introduction
According to the May 2020 announcement by the
PIA Research Institute, the number of live events,
including sports, concerts, and theater, canceled or
postponed from February 2020 to the end of that year
in Japan is estimated to be about 432,000, and theresulting financial loss due to those cancellations isestimated to be 77% of the market size, about 690
billion yen [1]. The amazing experience and cultureof sports and live entertainment, through which a
multitude of fans gather at a single venue and createa sense of unity by cheering loudly, is being forced tochange due to the novel coronavirus pandemic. During special session 1 of NTT R&D Forum 2020 Connect, the participants discussed the challenges andfuture vision concerning sports and live entertainment in the post-coronavirus era as well as the rolethat NTT should play.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

2. Initiatives by NTT toward thepost-coronavirus era
At the beginning, I explained what is happening inthe sports- and live-entertainment industries. Many
live-entertainment events have been canceled or postponed, and even when an event is held with no spectators or a reduced number of them, cheering loudly
and high-fiving are prohibited. Under such13

Feature Articles

circumstances, online live concerts were successfully
held in Japan and abroad by applying creativity andtechnology. Initiatives were taken to enliven sports
venues with no spectators by deploying “virtual fans”
via computer graphics and electronic-conferencing
systems, and bicycle racing went virtual by using asimulator.
I also introduced a technology called “Kirari!” that
NTT has been developing since before the pandemic,
which delivers the entire competition space to remotevenues in real time by transmission of ultra-realistic
information. A demonstration of distributed ultrarealistic viewing, which creates a sense of unity bysynchronizing the match venue and remote venues
with ultra-low-latency communication technology,
was showcased at the session.
3. Initiatives by the Japan Fencing Federation
Guest Yuki Ota, president of the Japan Fencing
Federation and vice president of the International
Fencing Federation, introduced the efforts both fencing federations have made from before the pandemic
to the present. The reform of fencing competitions,which has been ongoing since 2017, was described by
Mr. Ota. He talked about how the number of spectators and the unit price of tickets have increased bycreating posters to show the coolness of the fencers
and changing the venue from conventional gymnasiums to theaters.
He also reported on the remote-watching experience at a finals tournament held with the cooperationof NTT. The “Heartbeat Experience” is a system thatallows families of fencers supporting from a separate
venue to watch bouts while feeling the heartbeat of
the fencers during the bout. Data from a heart-ratemonitor worn by a fencer is transmitted to a ballshaped device possessed by the fencer’s family, and
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

the fencer’s heartbeat is experienced as “vibrationand light.” “Remote High Five” is a solution thatenables fencers at a tournament venue and theirfamilies cheering from another venue to celebratewith high fives. The vibration generated by a person
touching a transparent board installed at each venue
is measured and transmitted to a remote location
together with videos. This solution enables two-way
communication even when watching a bout from aremote location in a manner that communicates theexcitement and joy of sports.
4. Initiatives by IMAGICA EEX
The other guest, Haruyuki Moroishi, CEO andCCO of IMAGICA EEX Inc. and general producer of
IMAGICA GROUP, explained the transformation of
the live-entertainment industry’s business model during the pandemic. The industry has been growing
with the principle of “the supremacy of the realworld,” but from the perspective of avoiding the“three-Cs” (closed spaces, crowded places, andclose-contact settings) and maintaining social distance, this must be changed. He said it seems to beingtransforming from a “theater-oriented” businessmodel based on earning ticket revenue in the physicalspace of theaters, etc. to an “experience-value-oriented” business model based on using distribution andlive viewing services.
As his latest initiative, he introduced “Tsuyoshi
Nagabuchi Online Live ALLE JAPAN,” which he
was in charge of planning and producing. It was a live
performance, watched by more than 100,000 people,of 16 songs delivered online for two and a half hours.
By installing three 300-inch light-emitting diode(LED) screens, a 360-degree immersive stage environment was created. On these LED screens, 300
remote audiences were shown, who sang together,14

Feature Articles

which gave them a sense of unity and real-time experience.
At a normal live venue, there is a spatial distance
between the artist and fans due to physical constraints; however, through this new production andexpression of communication that uses video and live
streaming and application of new technology, thedistance of mind between them could be reduced tozero. Mr. Moroishi said that he discovered the possibility of new live entertainment that fuses the real andvirtual through this live event.
5. Discussion
While the effectiveness of online live events was
being reaffirmed, I asked the guests during the final
section of the session about the future of sports andlive entertainment. In response, while predicting that
future of such events will be a hybrid of online andreal events, Mr. Ota expressed a sense of crisis thatthe oligopoly of a few major event production companies may strengthen as globalization progresses. Mr.
Moroishi said that he is looking forward to the creation of new business and the expansion of the entertainment world resulting from the fusion of technology and creativity enabled by the spread of 5G (5th-generation mobile communication system), artificial
intelligence, and other new technologies. The discussion was very meaningful and lively on the topic offuture sports. Unfortunately, there was not enoughtime to continue the discussion.
Reference[1]

Press release issued by PIA Corporation on May 29, 2020 (in Japanese).
https://corporate.pia.jp/news/detail_covid-19_damage200529.html

Shingo Kinoshita

Executive Research Engineer, Supervisor, and Project Director, NTT Service
Evolution Laboratories.
He received a B.E. from Osaka University in 1991 and an M.Sc. with Distinctionin technology management from University College London, UK, in 2007. Sincejoining NTT in 1991, he has been engaged in research and development (R&D) ofdistributed computing systems, security, big data computing, and machine learning.
He was a senior manager of the R&D planning section of the NTT holding company
from 2012 to 2015, where he helped establish and operate NTT Innovation Institute,
Inc. in North America and managed R&D alliance and venture investments. He iscurrently in charge of the overall direction of various experimental R&D activities
for Tokyo2020 including assistance services for international visitors and entertainment services such as kabuki and those for SXSW (South by Southwest). He
received the 1998 DICOMO (Multimedia, Distributed, Cooperative, and Mobile)
Symposium Best Presentation Award, the 2003 CSS (Computer Security Symposium) Best Paper Award, the 2005 IPSJ R&D Award from the Information Processing Society of Japan, the 2016 Cool Japan Matching Award Grand Prix, the 2017
ACC New Technology Award from All Japan Confederation of Creativity (ACC),
the 2018 SPIKES ASIA finalist, the 2018 Digital Signage Award Grand Prix,
Cannes Lions Bronze, ONE SHOW Bronze, ACC Grand Prix, the 2019 ACC Silver,etc.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

15

Feature Articles: NTT R&D Forum 2020 Connect
Special Sessions

Challenges of NTT Space
Environment and Energy
Laboratories in the Coming Space
Millennium
Yuji Maeda
Abstract

On November 20th, 2020, special session 2 of NTT R&D Forum 2020
Connect was held and live streamed. The session welcomed novelist Yuya
Takashima, the science fiction consultant for “Mobile Suit Gundam THE
ORIGIN” and the multitalented entertainer Kaori Manabe as guests. The
author, Yuji Maeda, vice president of NTT Space Environment and Energy Laboratories moderated the session on the theme “Challenges of NTT
Space Environment and Energy Laboratories in the Coming Space Millennium.” Various research challenges tackled by the Laboratories, suchas optimal operation technology for fusion reactors, were introduced
while referring to the world in which the science-fiction work Gundam isset.
Keywords: space, energy, environmental issue

1. Introduction
NTT Space Environment and Energy Laboratories
was newly established as part of the Information Network Laboratory Group in July 2020. Under the basic
concept of “Beyond our planet. Beyond our future.”,we aim to contribute to the regeneration of the global
environment and creation of a sustainable and inclusive society. To this end, we are conducting research
to develop next-generation energy technologies, suchas nuclear fusion and space solar power systems,
technologies that enable resilient environmental
adaptation, and technologies to help achieve zeroenvironmental impact.
More specifically, the Laboratories includes the following groups: the Next-Generation Energy Technology Group, which researches overwhelmingly clean
energy, such as fusion reactors and space solar power
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

systems, as part of the Zero Environmental Impact
Research Project; the Energy Network Technology
Group, which researches distribution systems of thegenerated energy; the Sustainable Systems Group,
which researches sustainable systems that reduce
emissions of carbon dioxide; the ESG Management
Science Technology Group, which scientifically analyzes environmental, social, and governance (ESG)
management for predicting the future as part of the
Resilient Environmental Adaptation Research Project; and the Proactive Environmental Adaptation
Technology Group, which researches physically
adapting ourselves to the environment and controlling the climate.

16

Feature Articles

2. “Space Millennium” and the worldof Gundam
The first theme of special session 2 of NTT R&D
Forum 2020 Connect was “The ‘Space Millennium’
and the world of Gundam.” I asked the question,“Will Gundam ‘mobile suit’ be necessary for the
Earth in the future?” In response, novelist Yuya
Takashima, who was involved in the Japanese manga
and animation “Mobile Suit Gundam THE ORIGIN”
as the science-fiction consultant, cited “mobile workers” as an example. As the predecessor of the “mobile
suit,” mobile workers, which appeared in “Mobile
Suit Gundam THE ORIGIN,” are humanoid machines
created for lunar development work and can be considered highly advanced versions of a powered suit(also known as powered exoskeleton) currently being
developed in the actual world. He said, “I personallythink that it would be beneficial if we had something
like mobile workers that enable us to move freely inouter space.”
Multitalented entertainer Kaori Manabe pointed
out the prospect that “If virtual reality technology
becomes more advanced, there will be advantages
that exceed the five senses.” She also said, “Mobile
suit is supposed to be worn by humans, however, if
communication technology develops, it will be possible for people to not only go into space for real but
also, for example, go there by feelings only.” I
responded, “A world in which robots can be sentalone into outer space and be operated from the Earthis fantastic.” and concluded, “We are doing our best
to complete our mission to eliminate the world inwhich communication is not possible, including
space communication.”
After that, as a problem facing the current global
environment, I pointed out that the population of the
Earth has already increased too much and is expected

to increase further, and asked their opinion about the
migration of humans to a “space colony” as a realistic
solution. Responding with “I want to go, too!”, Mr.
Takashima said, “I have an image of staying in aspace colony for the long term, which is depicted in
Gundam.” In response, Ms. Manabe said, “Although
we assume that people in a space colony will live thesame life as those on Earth, it may be possible, forexample, for the body only to exist in the space colony and the mind to live in a virtual world that differs
in terms of life and consciousness.”
3. The “Space Millennium” and global
environmental issues
The second theme was “The Space Millennium andGlobal Environmental Issues.” I started by introducing the challenges NTT Space Environment andEnergy Laboratories is undertaking. We are currently
researching optimal operation technology for fusion
reactors in collaboration with the International Thermonuclear Experimental Reactor (ITER) and the
National Institutes for Quantum and Radiological
Science and Technology. To extract energy from afusion reactor, it is necessary to generate a plasma
and keep it stable for a long time. Therefore, it is necessary to transfer a huge amount of sensor data froma reactor to the control center, calculate optimum
values, and provide instant feedback to the reactor. To
implement that process, it is essential to further
increase the speed and reduce the latency of the network. Accordingly, NTT wants to contribute to this
implementation by using Innovative Optical andWireless Network (IOWN) technology.
4. Space solar power systems
We are also researching space solar power systems,with which energy generated in geosynchronous orbit
is transmitted over 36,000 km to the ground. I think if
the laser used for that power transmission is developed, it could also be used to control the weather bywarming and cooling clouds and the oceans. We aim
to roll out the laser in the real world after carrying out
various simulations on the Earth reproduced in cyberspace by using Digital Twin Computing technology
and confirming that the laser has no adverse effects
on the Earth or people.
5. Concluding remarks
Finally, I asked both guests about what they would

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

17

Feature Articles

request NTT Space Environment and Energy Laboratories to pursue. Mr. Takashima requested that we
conduct research on warp technologies. I replied,“It’s a different story when it comes to transporting
the human body, but I think warp will be a technology
in the field of communications,” leaving room for the

possibility of the Laboratories entry into research onthat topic. Ms. Manabe requested “to continue
researching in directions that enrich the natural worldby having both the values of a new technology andnature.”Yuji Maeda

Vice President of NTT Space Environment and Energy Laboratories.
He received a Ph.D. in systems information science from Future University Hakodate, Hokkaido, in 2013. He joined NTT Telecommunication Networks Laboratories in 1991. He was engaged in managing projects related to general emergency
management such as those concerning natural disaster response and cybersecurity.
He received the Scholarship Encouragement Award from the Institute of Electronics,
Information and Communication Engineers (IEICE) in 1998. He is a senior memberof IEICE and a member of the Institute of Electrical and Electronics Engineers(IEEE).

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

18

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Toward Cyber-physical Interaction
for Natural Connection of Real Space
and Cyberspace
Shigekuni Kondo, Atsushi Sagata, Kenichi Minami,
and Akihito Akutsu
Abstract

To actualize the concept of IOWN (Innovative Optical and Wireless Network), we need a new “environment” for connecting real space and cyberspace that fundamentally changes our lives without needing to be literate in information and communication technology. Also essential is a natural means ofincorporating that environment into our daily lives, which is to say a function for natural cyber-physicalinteraction. The following Feature Articles in this issue introduce the most recent technological trends
concerning cyber-physical interaction at the boundary between real space and cyberspace.
Keywords: perception, body, cybernetics

1. Introduction
In fusing real space and cyberspace by using theInnovative Optical and Wireless Network (IOWN)[1], we can expect more precise simulations that will
enable better predictions, thus expanding the range ofhuman activities. In such an environment where there
is a fusion of real space and cyberspace and our lives
are fundamentally changed, it will be possible foranyone, regardless of their information and communication technology (ICT)-literacy level, to benefit
from prediction. To achieve this, we believe it is essential to have natural cyber-physical interaction thatinvolves a natural means of information presentation
for fusing our daily activities with the environment.
The Feature Articles in this issue introduce the mostrecent trends and technologies that concern cyberphysical interaction at the boundary between real
space and cyberspace.
2. Overview of R&D on cyber-physicalinteraction
Research and development (R&D) on the fusion of
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

real space and cyberspace is already progressing to
the stage of practical application in many areas, andvarious types of content have been produced. In computer gaming, for example, game characters are displayed on smartphone screens and superimposed inreal space in front of the user, creating the illusion of
the characters existing in real space. There are also
many games that immerse the user in game worlds.
Regarding sports, there are, for example, online
bicycle races.
In the future, it will be possible for people to jump
into a virtual world (full-dive) and interact with real
space through cyberspace. For example, people canshare realistic places with others even if no one isactually there or provide realistic support for people
who are not at that location, amplifying humanknowledge and using human abilities to the maximum extent.
We believe that the user interface will play an evenmore important role than before in a future where real
space and cyberspace are tightly coupled. Technology for information presentation and input in ways that
do not interfere with human activities, new interaction technology that uses haptic sensory effects,
19Feature Articles

technology for making the utmost use of humanmotor functions, and other such technology areneeded for natural integration of people and the environment through natural information presentation,
which is to say natural cyber-physical interaction.
Cyber-physical interaction expands and develops
individual environments by connecting several environments and exchanging well-being and other subjective information as well as objective information,such as efficiency, quality, and cost, between them. A
core technology for such interaction provides perception and cognition control.
NTT laboratories will expand R&D on perception
and cognition and focus on R&D in the field of cybernetics on the basis of physiology.
3. Current work on cyber-physical interaction
The following Feature Articles in this issue introduce control technology for perception and cognition, which is a core technology for the cyber-physical interaction that is a subject of current R&D by
NTT laboratories.
“Improving Depth-map Accuracy by Integrating
Depth Estimation with Image Segmentation” [2]introduces a system called HiddenStereo that enables
natural three-dimensional (3D) viewing from monocular 2D images. This system is implemented by
combining technology for improving the accuracy ofdepth maps, which represent the distance from thecamera to each pixel in the image, and division of theimage into regions.
“Affect-perception Control for Enhancing a Senseof Togetherness for Remote Spectators” [3] introduces elemental technology for going beyond simply
transmitting and reproducing a sense of presence atan event venue to be experienced by remote viewers.
This technology also captures the emotional responses (emotional actions) of the remote audience andcreates a feeling of shared togetherness, interaction,and excitement through emotional feedback.
“Visible-light Planar Lightwave Circuit Technology and Integrated Laser-light-source Module forSmart Glasses” [4] introduces an ultra-compact RGB(red, green blue) laser-light-source module sized to fit
into the temples of smart glasses. The module is
implemented with an optical system that bundles
light sources that produce the three primary colors oflight (RGB) with a circuit that is drastically reducedin size.
“Fine-grained Hand-posture Recognition for Natural User-interface Technologies” [5] introduces
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

research for establishing finger-shape recognition
technology to implement operation of smart glasses
through hand gestures.
“Information-display Method for Reducing Annoyance by Gaze Guidance” [6] introduces an information-display method that both reduces the user’s feeling of annoyance and increases the certainty of
information access by using an imposed display
technique that gives the user the feeling that the act ofreading information was their own choice.
“Presenting Material Properties with Mid-air Pseudo-haptics” [7] introduces a mid-air pseudo-haptic
technology that gives the user the perception of thematerial properties of virtual objects as the user
manipulates them through their own action.
“Evaluation of Adaptability to Unfamiliar Environments Using Virtual Reality” [8] introduces researchon creating technology for evaluating and improving
the ability to adapt to the environment to achieve
appropriate exercise and prevent accidents involving
the elderly while walking or driving.
4. Conclusion
We described the R&D on technology for cyberphysical interaction at the boundary between real
space and cyberspace, particularly the most recent
research on perception and cognitive control. We arealso investigating other core technologies for cyberphysical interaction, including technology for physiological control, emotion and desire control, communication of the five basic and other human senses,
communication control, and social capital infrastructure.
Toward making IOWN a reality, NTT laboratories
have been working to bring on-going work related toperception and cognition to maturity and will continue to promote R&D on cyber-physical interaction
as an unprecedented user interface based on humanproprioception for a seamless connection of bodies incyberspace and those in real space in the field ofcybernetics.
References
[1]
[2]

[3]

NTT Technology Report for Smart World 2020, https://www.rd.ntt/e/techreport/
M. Ono, Y. Kikuchi, T. Sano, and S. Fukatsu, “Improving Depth-map
Accuracy by Integrating Depth Estimation with Image Segmentation,” NTT Technical Review, Vol. 19, No. 3, pp. 22–26, Mar. 2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa4.html
T. Sano, M. Makiguchi, H. Nagata, and H. Seshimo, “Affect-perception Control for Enhancing a Sense of Togetherness for Remote20Feature Articles

[4]

[5]

Spectators,” NTT Technical Review, Vol. 19, No. 3, pp. 27–30, Mar.
2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa5.html
T. Hashimoto and J. Sakamoto, “Visible-light Planar Lightwave Circuit Technology and Integrated Laser-light-source Module for Smart
Glasses,” NTT Technical Review, Vol. 19, No. 3, pp. 31–36, Mar.
2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa6.html
Y. Kubo, “Fine-grained Hand-posture Recognition for Natural Userinterface Technologies,” NTT Technical Review, Vol. 19, No. 3, pp.
37–39, Mar. 2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa7.html

Shigekuni Kondo

Senior Research Engineer, Planning Section,
NTT Service Evolution Laboratories.
He received a B.E. in applied physics and M.S.
in integrated design engineering from Keio University, Kanagawa in 2003 and 2005 and joined
NTT in 2005. His research interests include
software modularity, home network, and userinterfaces. He is a member of the Institute ofElectronics, Information and Communication
Engineers (IEICE).

[6]

[7]

[8]

R. Saijo, T. Sato, S. Eitoku, and M. Watanabe, “Information-display
Method for Reducing Annoyance by Gaze Guidance,” NTT Technical
Review, Vol. 19, No. 3, pp. 40–44, 2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa8.html
T. Kawabe, “Presenting Material Properties with Mid-air Pseudohaptics,” NTT Technical Review, Vol. 19, No. 3, pp. 45–48, Mar.
2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa9.html
T. Isezaki and T. Watanabe, “Evaluation of Adaptability to Unfamiliar
Environments Using Virtual Reality,” NTT Technical Review, Vol. 19,
No. 3, pp. 49–52, Mar. 2021.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202103fa10.html

Kenichi Minami

Director of Research, Planning Section, NTT
Service Evolution Laboratories.
He received a B.E. in electronic engineeringand M.S. in biomedical engineering from Keio
University, Kanagawa in 1991 and 1993. Hereceived an MBA from Thunderbird, Global
School of Management, Arizona, USA, in 2002.
He has been engaged in R&D management in thedevelopment of “Kirari!” immersive telepresence technology since 2016. From 2012 to 2014,
he was responsible for the development ofmobile application services at NTT DOCOMO.
His research interests include image and audio
processing, user interfaces, and telepresencetechnologies. He is a member of IEICE.

Atsushi Sagata

Executive Research Engineer, Cybernetic
Intelligence Research Project, NTT Service Evolution Laboratories.
He received a B.E. in electronic engineeringfrom the University of Tokyo in 1994 and joined
NTT the same year. He has been engaged in
R&D of video coding systems and in the development of the digital high-definition TV transmission system at NTT Communications. He is amember of IEICE.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Akihito Akutsu

Vice President, NTT Service Evolution Laboratories.
He received an M.E. in engineering from Chiba
University in 1990 and Ph.D. in natural scienceand technology from Kanazawa University in
2001. Since joining NTT in 1990, he has been
engaged in R&D of video-indexing technology
based on image/video processing and manmachine interface architecture design. From2003 to 2006, he was with NTT EAST, where he
was involved in managing a joint venture
between NTT EAST and Japanese broadcasters.
In 2008, he was appointed director of NTT Cyber
Solutions Laboratories (now NTT Service Evolution Laboratories), where he worked on an
R&D project focused on broadband and broadcast services. In October 2013, he was appointed
as executive producer of 4K/8K HEVC (High
Efficiency Video Coding) at NTT Media Intelligence Laboratories. He received the Young
Engineer Award and Best Paper Award fromIEICE in 1993 and 2000, respectively. He is amember of IEICE.

21

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Improving Depth-map Accuracy byIntegrating Depth Estimation with
Image Segmentation
Masato Ono, Yumi Kikuchi, Takashi Sano,and Shinji Fukatsu
Abstract

NTT Service Evolution Laboratories is conducting research and development on new media-processing technologies to enable natural interaction by presenting information that influences the visual senses.
This article introduces our current research and development of a media-processing technology for
improving the accuracy of depth maps by combining depth estimation with image segmentation and asystem that uses this technology to generate HiddenStereo (a technology developed by NTT Communication Science Laboratories) images, which enables natural three-dimensional viewing from monocular
two-dimensional images.
Keywords: 3D images, video streaming, depth maps

1. Integrating depth estimation with image
segmentation to improve depth-map accuracy
Depth maps are a representation of the distancefrom the camera to the subject for each pixel in animage. They have various applications, including thefamiliar smartphone camera function that blurs distant background images when taking a picture anddetecting nearby objects for self-driving vehicles.
We are conducting research and development(R&D) on a media-processing technology for
improving the accuracy of depth maps by converting
two-dimensional (2D) content into 3D content inaddition to effectively representing newly created 3D
content in the entertainment field (Fig. 1). This technology is composed of depth-map-generation technology, which generates accurate depth informationfrom 2D images, and depth-map-optimization technology, which corrects depth maps for effective 3D
representation and other purposes.

2. Technical points(1) Depth-map-generation technology
There are various methods for obtaining depth
maps, such as using the parallax between images
from stereo cameras, combining camera images withinformation from a separate device (LiDAR*, etc.), or
using various image-processing techniques. Deeplearning has also been used recently to generate depth
maps from 2D content, but the depth maps are generally of low resolution, and depth maps with clear
outlines of the subjects cannot be obtained. Whenapplied to recent high-definition (4K/8K) images, theresolution and quality of a depth map must also be
high.
Regarding edge-preserving smoothing [1, 2], weare developing a technology for correcting depth
maps, improving their accuracy, and enabling effective 3D representation by defining clear outlines foreach subject in an image and performing edge-preserving smoothing on the results of image segmentation*

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

LiDAR: light detection and ranging.

22

Feature Articles

Depth-map-optimization
technology

Depth-map-generation
technology
Panoptic
segmentation
Depth estimation

Depth map after
optimization

Depth map
Edgepreserving
smoothing

Monocular 2D
image

Depth-map
optimization

3D
conversion
process

3D image
Generate highly accurate
depth map

Depth-map correction for
effective 3D representation

Conversion toeffective 3D content

Fig. 1. R&D of media-processing technology for improving the accuracy of depth maps.

Extract subjects and background
Subject edge preservation

Segmentation result

Depth map afteredge-preserving
smoothing

Original image

Depth-estimation result

Fig. 2. Depth-map-generation technology.

(Fig. 2). Specifically, the resolution of a depth map
can be increased by filtering the low-resolution depth
map obtained though depth estimation using the segmentation-result image as a guide, while also preserving the edges of the subjects in the image. We use
the results of panoptic segmentation, which is a combination of semantic segmentation and instance segmentation, to enable separation of subjects and thebackground.
With this technology, depth estimation and segmentation are only loosely related, so the algorithms
for each can be adjusted as necessary. For example,
we are currently using deep-learning algorithms forboth depth estimation and segmentation, but otheralgorithms, such as those including self-supervised
learning, can easily be substituted.
(2) Depth-map-optimization technology
The depth-map-generation technology describedabove can provide accurate depth maps, but whenused as is, the maps may not necessarily be suitable
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

for a particular application. For example, if the depth
in the image is simply reduced to the depth range thatcan be expressed as a depth map, the depth differencein the image will not be sharp and the 3D expression
will not be effective.
As a consequence, methods are used to correct
depth maps to improve the sense of presence whenviewing the 3D video. We are also developing a technology for optimizing depth maps, emphasizing thesense of depth surrounding the subject(s) that are thefocus, to achieve a more effective 3D effect (Fig. 3).
In particular, corrections are made from the following
two perspectives.
(a) Limiting the range of depths used in 3D representations
A perceptual tendency when viewing in 3D is that
it is difficult to distinguish fine depth differences inareas that are distant in the depth direction from thesubject of focus (i.e., far in the background or close in
the front), so we reduce the range of depth differences
23

Feature Articles

Depth map
after optimization

Depth map
before optimization

Optimization

Apply
Histogram

Histogram

Mapping function

Output

Back

Input

Front

(a) Depth range used in the 3D
representation is limited to therange of the subject that is thefocus.

Back

Front

Within the limited depth rangethat can be represented, convert
a depth map into a depth mapthat effectively represents a
sense of depth for the subject.

(b) Correction emphasizing
differences in depth

Notes

Depth layers containing
the subjects of focus

Fig. 3. Depth-map-optimization technology.

allocated to those depth ranges. Specifically, we analyze a depth map represented in steps from 0 to 255,
create a histogram, and define ranges containing the
subject as effective depth ranges. For example, this
could be the range from the 5th percentile to the 95th
percentile in the distribution of depth values from thedepth map. Depths within this range are then extended over the range from 0 to 255, and values outside
this range are mapped to either 0 (farthest) or 255(nearest). Therefore, the 3D representation only
expresses the effective depth range for 3D viewing.
(b) Representations emphasizing depth differences
By emphasizing the sense of depth in the range near
the subject of focus, 3D video representing more
subtleties can be generated. Specifically, from thedepth-map histogram, we derive depth layers fromthe depth ranges containing the subjects of focus. The
range of the depth layer with the subject to be emphasized most is then expanded, and the positions anddepths of each of the depth layers are adjusted toemphasize the depth representation of that layer. This
generates a mapping function for depth values, with
input depth values on the horizontal axis and output
values on the vertical axis, which has a segment for
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

each depth layer. The segment for the layer containing the most emphasized subject has the greatest
slope.
On the basis on the above two points, we derive a
mapping function (nonlinear depth-map transform)
to correct the depth map and apply it to the depth-map
image, generating an optimized depth-map image.
There are various types of 2D archive video containing a range of camera work and scene changes, so
the content of depth maps tends to change greatly
with time. To enable depth-map optimization that can
produce effective 3D representations for all suchcases, we intend to continue applying our depth-mapoptimization technology to various types of content,
evaluate the results, and improve the quality of depthmap optimization.
3. Use in HiddenStereo
We are currently promoting the development of asystem to generate HiddenStereo (a technology
developed by NTT Communication Science Laboratories [3]) images, which enables natural 3D viewing
using depth maps generated and optimized using all24

Feature Articles

Display 3D/2D simultaneously
at movie theaters, Karaoke boxes.
HiddenStereo
3D imagegeneration system

Monocular 2D image

HiddenStereo
content

2D image

Project disparity-inducer
pattern onto 2D work

Various types ofarchive images(2D, 3D-stereo)Group viewing

Display
HiddenStereo
images3D viewing

Ordinary 2D viewing

Freely selectable/switchable

Exhibition at art gallery or museum
3D viewing
View original picture

HiddenStereo
processing(for left eye)

Depth map after

Monocular 2D image

Technology for
optimization
improving depth-map
accuracy by integrating
depth estimation withimage segmentation

Create disparityinducer pattern

HiddenStereo image(for right eye)Fig. 4. HiddenStereo.

the above-mentioned technologies (Fig. 4).
HiddenStereo generates a disparity-inducer patternfrom the original video and depth map, which provides depth information to human viewers. Images
for the left and right eyes can be generated by adding
or subtracting this pattern from the original image.
When the left and right images are added together, thedisparity-inducer patterns completely cancel eachother out, leaving the original image, so that viewers
without 3D glasses can see the 2D image clearly.
Viewers with 3D glasses see the image with depth
due to the effects of the disparity-inducer pattern.
This technology enables viewer-friendly 3D display, allowing each viewer to select how they want toview at any time, and all view the same display content. Since no special equipment is needed to display
2D and 3D at the same time, existing 3D display
environments can be used as-is. Operational requirements for ordinary 3D video projection, such as presenting 3D and 2D video at different times, or preparNTT Technical Review

Vol. 19 No. 3 Mar. 2021

ing separate venues for 3D and 2D presentation are
no longer necessary, which could reduce the operating costs for 3D video presentation.
By using our technology for improving depth-map
accuracy, by integrating depth estimation with image
segmentation, with HiddenStereo, we can convert not
only new content captured with stereo cameras orcreated with 3D computer graphic production but
also from 2D content produced in the past into 3D
content.
4. Future developments
We introduced a media-processing technology weare developing for improving the accuracy of depth
maps by integrating depth estimation with image segmentation, and a system using this technology to
generate HiddenStereo images, enabling natural 3D
viewing from monocular 2D image. We will continue
investigation to further improve the speed and quality
25

Feature Articles

of each of the component technologies throughverification trials and other means to contribute toimplementing businesses using natural interaction.
References[1]

K. He, J. Sun, and X. Tang, “Guided Image Filtering,” 11th European

Masato Ono

[2]
[3]

Conference on Computer Vision (ECCV), Heraklion, Crete, Greece,
Sept. 2010.
G. Pestschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, andK. Toyama, “Digital Photography with Flash and No-Flash Image
Pairs,” ACM Trans. Graph., Vol. 23, No. 3, pp. 664–672, 2004.
T. Fukiage, T. Kawabe, and S. Nishida, “Hiding of Phase-based Stereo
Disparity for Ghost-free Viewing Without Glasses,” ACM Trans.
Graph., Vol. 36, No. 4, pp. 147:1–17, 2017 (Proc. of SIGGRAPH
2017, Los Angeles, USA).

Takashi Sano

Research Engineer, Natural Communication
Project, NTT Service Evolution Laboratories.
He received an M.E. in information engineering from University of Tsukuba, Ibaraki, in 2006.
He joined NTT EAST the same year and engaged
in network engineering, customer service management, and creating business-to-business services. He moved to NTT Service Evolution
Laboratories in 2016 and is developing new services based on R&D products.

Senior Research Engineer, Innovative Service
Research Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in electronicengineering from Tokyo University of Science in2002 and 2004. In 2004, he joined NTT CyberSpace Laboratories. He has been engaged in
R&D on software and hardware design of videocoding.

Yumi Kikuchi

Shinji Fukatsu

Research Engineer, Natural Communication
Project, NTT Service Evolution Laboratories.
She received a B.E. and M.E. in electricalengineering from Tokyo University of Science in2000 and 2002. Since joining NTT in 2002, she
has been researching and developing contentconfiguration methods and content-delivery systems. Her present research is focused on featurepoint extraction of images and matching using
the extracted feature points.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Senior Research Engineer, Supervisor, Innovative Service Research Project, NTT Service
Evolution Laboratories.
He received a Ph.D. in engineering from Osaka
University in 2002 and joined NTT the same
year. He has been engaged in R&D of humaninterfaces and video streaming services. He has
also been engaged in planning and developmentof video streaming services at NTT Plala and inpromoting standardization and international
development of information and communication
technology at the Ministry of Internal Affairs andCommunications. He is currently engaged in
R&D of remote-world infrastructure technology.

26

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Affect-perception Control for
Enhancing a Sense of Togetherness
for Remote Spectators
Takashi Sano, Motohiro Makiguchi, Hidenobu Nagata,and Hitoshi Seshimo
Abstract

The COVID-19 pandemic has affected the world of sports and entertainment by expanding the audience for spectator-less sports broadcasts and live-event distribution. In addition to the research NTT
Service Evolution Laboratories has been conducting on remote transmission and reproduction of senseof-presence for event venues, we are promoting research on elemental technology for capturing theemotional movements (affect) of remote viewers to provide what we believe is the emotional feedback
needed for such viewers to share a sense of togetherness, interactivity, and excitement as spectators.
Keywords: affect, sense of togetherness, enthusiasm

1. BackgroundAs the COVID-19 pandemic is currently restricting
personal movement and face-to-face interaction,
there has been rapid progress in creating remoteforms of various human activities. For sports andentertainment, the broadcasting and live distributionof spectator-less events is increasing. However, it is
not yet possible for remote spectators to experience
the intensity and extraordinary feeling of being at a
stadium or live venue.
NTT Service Evolution Laboratories has been conducting research and development (R&D) on technology for sensing the entire space of an event venue
and transmitting and reproducing the on-site senseof-presence and audience enthusiasm to remote liveviewing locations. The technology includes advanced
real-time object extraction [1], ultrawide video composition [2], Advanced MMT *1 for synchronoustransmission of video and audio [3], and glassless
three-dimensional video display that applies the principles of perceptual psychology [4]. However, participation at stadiums and live venues is restricted,
and remote sites are not large like live-viewing locaNTT Technical Review

Vol. 19 No. 3 Mar. 2021

tions. In a situation where the viewing site is shifting
to small, independent locations, such as the home, it
is not sufficient to simply convey the enthusiasm andintense moments of the actual site. Instead, a sense of
togetherness and interaction among remote spectators can be used to enhance the sense of excitement.
We are therefore beginning R&D on affect-perception control for capturing the emotional responses ofviewers using simple devices. By establishing elemental technology for that purpose, we intend toenable an experience in which even remote viewers
can feel the thrill of watching an event together withmany people in a stadium or live venue.
2. Affect-perception control
Affect-perception control can be broadly categorized into three types of technology: field*2-sensing*1 Advanced MMT: MPEG Media Transport (MMT) is a media
transport standard developed by the Moving Picture Experts
Group (MPEG), a working group of International Organizationfor Standardization/International Electrotechnical Commission(ISO/IEC). Advanced MMT is highly realistic media streaming/
synchronization technology developed by NTT.

27

Feature Articles

Live
broadcastviewing

Remote-world platform
Ultra-reality field-generation technology

Venue

Remote audience

Real-worldmodeling function

Virtual-audiencespace

Low-latency telecommunication
Field-sensing technology

Field-reproduction technology

Multi-channel
sensing function

Zero-latency
feedback function

Sensing device

Audio visual andhaptic devices(external operation)

Remote viewing network

Remote viewing assistance terminal

Clarification of feedback stimuli that canconvey changes in human affectFig. 1. Overview of affect-perception control.

technology for synchronous detection of the remote
viewer’s reactions and local situation, ultra-reality
field-generation technology for creating a virtualaudience-space model for virtual relocation of a large
number of remote viewers from any location in a
spatially natural manner, and field-reproduction technology for zero-latency feedback of changes in theaffect of other viewers in response to each other’sstates in the virtual-audience space (Fig. 1). These
technologies are explained in the following sections.
2.1 Field sensing
Even when watching the same sports event, somepeople will follow the movement of a particular
player while others are analyzing the overall situationfrom a wide view and still others are cheering in theaudience around them. Thus, what people are paying
attention to and the timing of changes in spectator
affect differ from person to person. This is also true
for other live entertainment events, e.g., when different audience members get excited at different songs
or performances. The objective of our research is toachieve real-time sensing, visualization, and quantification of human affect with multi-channel sensing
implemented by interworking among multiple sensing devices such as voice-acquisition devices (e.g.,smart speakers, television (TV)-mounted cameras),person-sensing devices (e.g., air conditioners), andwearable devices (e.g., smart watches that measurevital signs).
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Specifically, cameras and motion sensors are used
to capture the viewing environment such as the position of the TV, pose of audiences, and line of sight ofthe viewer. Sensing is used to detect the degree ofviewer concentration and which part of the viewing
content has the viewer’s interest. Then, the viewing
content and changes in the viewer’s movements andvoice are analyzed to learn what types of situations
are associated with changes in viewer affect. Thesechanges in a viewer’s affect can be measured in moredetail by sensing internal body signals, such as bodytemperature and pulse, at the same time as measuringexternal changes such as movement.
2.2 Ultra-reality field generation
This technology implements a virtual-audiencespace function for real-world modeling that turns theviewer’s environment into a virtual-spatial model andfills the space with many remote viewers in a naturalmanner. Establishing this technology will make itpossible to aggregate the viewer-affect informationcollected with the technology described in the previous section and categorize viewers by affect to construct an audience space that enhances enthusiasm. It
may also be possible to increase the interest of viewers categorized as showing less interest in the content
when watching alone by placing them in an audience*2 In this article, “field” means the person’s affect and the spacearound him/her.

28

Feature Articles

Fig. 2. Experimental space.

space with highly enthusiastic viewers. Analysis ofsports action and live content to synthesize virtual
cheering (fake crowds) according to the exciting parts
would make it possible to construct special spacesthat create the impression of audiences of tens or even
hundreds of thousands cheering in stadiums or event
venues that accommodate just a few thousand spectators or even no audience at all.

well as an optimal sensing and feedback method. We
did so by virtually expanding the living room into avirtual connected remote live-viewing venue wheremultiple spectators gather according to excitement in
the game and by presenting performances that produce a reaction by adding effects to video in accordance with the appearance of viewer emotion detected by sensing.

2.3 Field reproduction
The data for the virtual space described above are
combined with information that creates a feeling of
the presence of other viewers, such as video, sound,and shadows, to implement a zero-latency feedback
function that augments change in viewer affect andenthusiasm. To increase the sense of togetherness
among remote viewers by implicitly creating thesense of cheering together side-by-side, we are developing spatial-presentation technology that focuses ontouch and other senses in addition to visual and auditory senses. The timing of triggering affect control isalso important. Together with technology for bidirectional transfer of a huge amount of sensory datawith zero-latency, we are also engaged in R&D on
technology for autonomous presentation of information that suggests changes in affect by predicting thenext reaction either from changes in viewer affect orcontext changing in content at the viewer’s location
instead of transmitting all the sensory data to eachlocation.
To promote R&D on affect-perception control, weconstructed an experimental space to verify its effectiveness (Fig. 2). Taking the viewing of a sports eventin the living room at home as the use case, we verified
the degree of influence on change in viewer affect as

3. Future prospects

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

For future work, we will promote R&D for creating
new value in the world of remote activities by conducting verification experiments in various realworld situations such as viewing sports events andentertainment distribution with the objective of establishing affect-perception control.
References
[1]
[2]

[3]

[4]

H. Miyashita, K. Takeuchi, H. Nagata, and A. Ono, “Fast Image Segmentation for 4K Real-time Video Streaming,” IEICE Technical
Report, Vol. 117, No. 73, MVE2017-13, pp. 189–190, 2017.
T. Yamaguchi, T. Sato, M. Ono, Y. Tonomura, K. Namba, Y. Kikuchi,
T. Hoshide, T. Morizumi, A. Ono, and K. Minami, “Surround Video
Stitching and Synchronous Transmission System for Immersive Live
Viewing Services,” Journal of the Institute of Image Information andTelevision Engineers, Vol. 74, No. 2, pp. 402–411, 2020.
M. Ono, T. Hoshide, and S. Fukatsu, “Multiple Video, Audio, andLighting Control Signal Synchronous Transmission System Using
Advanced-MMT (MPEG Media Transport),” ITE Technical Report,
Vol. 44, No. 7, BCT2020-39, pp. 33–36, 2020.
M. Makiguchi, D. Sakamoto, H. Takada, K. Honda, and T. Ono,
“Interactive 360-degree Glasses-free Tabletop 3D Display,” Proc. ofthe 32nd Annual ACM Symposium on User Interface Software andTechnology (UIST 2019), pp. 625–637, New Orleans, USA, Oct.
2019.

29

Feature Articles

Takashi Sano

Senior Research Engineer, Innovative Service
Research Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in electronicengineering from Tokyo University of Science in2002 and 2004. In 2004, he joined NTT CyberSpace Laboratories. He has been engaged inresearch and development on software and hardware design of video coding.

Hidenobu Nagata

Senior Research Engineer, Supervisor, Innovative Service Research Project, NTT Service
Evolution Laboratories.
He received an M.E. in systems and information engineering from the Faculty of Engineering, Hokkaido University in 2001 and joined
NTT the same year. He studied video-handlingtechnologies including video search, video
indexing, automatic content summarization, andtheir interfaces. From 2008 to 2014, he worked at
NTT Electronics and developed professionaltranscoders and embedded audio for consumer
devices. He moved to NTT in 2014 and is currently researching ultra-realistic telecommunication technology.

Motohiro Makiguchi

Researcher, NTT Service Evolution Laboratories.
He received an M.A and Ph.D. in computerscience and information technology from Graduate School of Information Science and Technology of Hokkaido University in 2012 and 2020.
His major research interests include aerial display, auto-stereoscopic 3D display, and interaction based on 3D display.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Hitoshi Seshimo

Senior Research Engineer, Supervisor, Innovative Service Research Project, NTT Service
Evolution Laboratories.
He received an M.E. in mechanical engineering from Waseda University, Tokyo, in 1997 andjoined NTT the same year. His research interests
include computer aided instruction, web-basedlearning, content distribution and navigation
systems, and geographical information services.

30

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Visible-light Planar Lightwave
Circuit Technology and Integrated
Laser-light-source Module for Smart
Glasses
Toshikazu Hashimoto and Junji Sakamoto
Abstract

NTT Device Technology Laboratories has applied planar lightwave circuit technology, which is used
to integrate optical circuits on a planar substrate for optical communication devices, for use in the visible-light domain, enabling the development of dramatically smaller optical systems using all three primary colors of light (RGB: red, green, and blue). We applied this technology to an ultra-compact RGB
laser-light-source module that is small enough to be installed on the temples of glasses, making smart
glasses feasible.
Keywords: smart glasses, visible-light planar lightwave circuits, laser-light-source module

1. Introduction
As information and communication technology has
progressed, new lifestyles are emerging like neverbefore through information, e.g., online distributionof music and video and chat services. This information is brought to us through information terminals
connected to networks. Various information terminals, such as personal computers, smartphones, andmore recently, devices such as smart speakers, arebeing used for many daily-life situations. However,
the information we receive is limited to where theterminal or other device is, and smartphones are not
an exception to this. With smartphones, our access to
information is possible only by peeking through asmall “window” (screen). If we could develop information terminals that we could use in a more naturalway, lifestyles would become richer and freer without
such limitations.
The optical circuit technology introduced in thisarticle is intended for application to smart glasses,
which will provide access to visual information in amore natural way, as discussed above. Smart glasses
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

are display devices that should feel comfortable, likeputting on ordinary glasses, and improve on current
goggle-shaped head-mounted display devices that are
being used for virtual reality, augmented reality,
mixed reality, and other applications, which are
generically called extended reality. Various levels ofdisplay performance are possible, but to provide an
immersive experience, as with augmented reality,
technology for providing high resolution and a wide
field of view is necessary. Having smart glasses look
like ordinary glasses requires the components to bevery small, so technology is needed to achieve both
optical characteristics and miniaturization. NTT
Device Technology Laboratories developed theabove optical circuit technology for dramaticallyreducing the size of optical systems that handle lightin the RGB (red, green, and blue) color space. This
technology is called the RGB coupler and is suitable
for the type of smart glasses describe above [1, 2].
This article introduces the RGB coupler and its application to an ultra-compact RGB laser-light-source
module for smart glasses.

31

Feature Articles

Trunk network

Wafer (PLC)~40 µmWavelength multiplexer/demultiplexer

10 cm−1 cm
Metropolitan
network

Core

HeaterON/OFF

ON
OFF

~7 µm
Cladding

Electron microscope images

Optical switch

Opticalsubscriber
network

Optical branch circuit

Fig. 1. PLCs and applications in optical fiber communications.

2. Visible-light planar lightwave
circuit technology
For optical fiber communications, NTT is conducting research and development on a component technology called planar lightwave circuits (PLCs)*, inwhich optical circuits are integrated on a silicon
substrate, similarly to electronic integrated circuits.
These optical circuits are fabricated using silica glass,the same material for optical fiber, and are composedof paths several microns wide called cores. The cores
have a higher refractive index than the surrounding
material, causing the light to follow them (opticalwaveguide structures). Circuits can be built to branch,interfere with, or control the phase of the light. PLC
technology is used in devices for optical fiber communications, such as optical splitters and wavelength
multiplexers/demultiplexers [3] (Fig. 1). Infraredlasers with wavelengths, such as 1.55 μm, are used for
optical fiber communications, so the laser technology
for such wavelengths and optical circuit technology
to manipulate them (e.g., wavelength multiplexers/
demultiplexers) have been developed. The wavelengths of visible light are in the range of 0.4 to 0.7
μm, and since mass production of green semiconductor lasers was achieved around 2009, it has been possible to use all three primary colors. Just as visiblelight light-emitting diodes came into general use over
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

a period of more than ten years, visible-light lasers
are expected to become common. When this occurs,
technology to provide stable optical systems will be
important for various applications using laser-lightproperties such as interference. NTT is applying PLC
technology in the visible-light domain to develop
such systems. Applications for this technology
include optical light sensing and optical quantuminformation processing. The RGB coupler is anotherapplication of this technology.
3. Retina-scanning laser eyewear and RGB
light source
Before describing the RGB coupler, we firstdescribe scanning laser eyewear. As the name suggests, laser eyewear refers to a glasses-type display
device that uses laser light [4]. Laser light has theuseful property of not diverging over distance, so thatjust like with a laser pointer, when a laser ray hits ascreen it makes a clear, bright spot, regardless of the
distance to the screen. By scanning this point of laser*

PLC: Optical integrated circuits fabricated using a technology
combining optical fiber manufacturing technology and semiconductor integrated circuit technology. Circuits are fabricated mainlywith quartz glass, so they are compatible with optical fiber materials (low optical losses when connected together). PLC is used
as a component technology for optical fiber communications.

32

Feature Articles

Human eyeRetinaMicromirror(MEMS)

Crystalline lens

RGB light source(conventional technology)
R

Concave mirror

G
B
Semiconductor
lasers

LensesDichroic mirrors

MEMS: microelectromechanical system

Fig. 2. Principles of retinal-projection laser eyewear.

light across the screen while changing the brightness,
a focused image can be displayed regardless of the
distance to the screen. Scanning laser eyewear applies
this principle to the human eye, using the retina as the
projection screen. The crystalline lens of the humaneye must change shape to form an image (focus) on
the retina, but with a scanning laser, a focused image
is projected on the retina regardless of the shape ofthe lens, so an image can be obtained regardless ofvisual acuity. This system is extremely simple, as
shown in Fig. 2, and composed of just microelectromechanical system mirrors that scan the laser lightsource and another mirror that directs the laser lightthrough the pupil. Note that the intensity of the laser
is adjusted to a safe level, which is roughly 1/1000
that of a laser pointer. However, current components
for such a configuration are not small enough forglasses-type laser eyewear. The laser-light-source
module is a particularly complex structure, involving
lenses or mirrors to combine the outputs from thethree (RGB) semiconductor lasers, and is difficult tominiaturize (Fig. 2, lower left).
4. RGB coupler and RGB
laser-light-source module
NTT Device Technology Laboratories has focusedon the lens and mirror structure as the optical components of the laser-light-source module describedabove and has significantly reduced the size of themodule by applying visual-light PLC technology
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

(Fig. 3(a)). Visible light has a short wavelength, lessthan half of those used for optical fiber communications, so we investigated all aspects of optical-circuit
elements. We developed a new wavelength multiplexer for the RGB coupler, which has the smallest
chip size yet achieved. Figures 3(b) and (c) respectively illustrate the operation of a conventional wavelength multiplexer and our new wavelength multiplexer. When two light paths (optical waveguides) arebrought together, light is transferred from one waveguide to the other due to a kind of resonance, and thisis called a directional coupler. The length of thedirectional coupler required to transfer signals
through resonance depends on the wavelength, so by
matching the period of the transition length, light canbe combined in a single waveguide (Fig. 3(b)). At
NTT, we have taken this a step further by placing a
waveguide of a different width between them andfound a width that simultaneously satisfies both resonant and non-resonant cases, depending on the wavelength (Fig. 3(c)). Using this configuration, we cantransfer only a specific wavelength of light to theother waveguide without the need to match the periodof the transition length, and light can be combined
into one waveguide in a distance approximately one
third of the conventional directional coupler. The
light transmittance of the RGB coupler is shown in
Fig. 4. The RGB wavelengths, shown in the figure
with dotted lines, have high transmittance, indicatingthat favorable characteristics can be achieved. A chipof the RGB coupler is shown in Fig. 5(a). The chip
33

Feature Articles

RGB light source(conventional technology)
R

Light-source configuration
with RGB coupler

G(a) Conventional light-source
configuration with laser-light-source
module and RGB coupler

PLC chipWhite light source

B
Directional coupler

R(b) Conventional wavelength
multiplexer

Optical waveguide

B+GR

Adjusting width creates opticalwaveguide that transmits only
specific wavelengths(c) New wavelength multiplexer
B+G

Fig. 3. Operation of conventional and new wavelength multiplexers.

B

0

G

R

Transmittance (dB)−2−4Input port
B
R
G

−6

−8

−10
400

500

600

700

Wavelength (nm)Fig. 4. RGB coupler transmittance.

size is 2.1×3.5×0.5 mm, and to the best of our knowledge, it is the smallest RGB coupler chip to date.
NTT Device Technology Laboratories has also collaborated with TDK Corporation to implement a
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

packaged optical light-source module (Fig. 5(b)).
From this collaboration, we were able to fabricate thesmallest RGB laser-light-source module (5.5×8×2.7mm) by combining the semiconductor laser chips on
34

Feature Articles(a) Compact RGB coupler chipWhite light

Size: 2.1×3.5×0.5 mm(b) Laser-light-source module

Size: 5.5×8×2.7 mm(c) For application in smart glasses

Fig. 5. Fabricated RGB coupler and laser-light-source module.

subcarriers with TDK’s technology for aligning andfixing the lasers and optical waveguide ends with anaccuracy of better than 1 μm in a very short time. The
size of this ultra-compact RGB laser-light-source
module is approximately one twentieth that of earlier
RGB light-source modules using lenses or mirrors. It
is comparable to the width of the temples of glasses
and is suitable for implementing glasses-type laser
eyewear, as shown in Fig. 5(c).
Large parts of the environment in which we live are
now being produced by information. As the smartphone has had a huge influence on our lives, thetechnology introduced in this article will contribute to
the development of display devices that can be usedanywhere and much more naturally. This will contrib-

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

ute to creating richer environments in which to live.
References
[1]

[2]

[3]
[4]

J. Sakamoto, G. Takashi, S. Katayose, K. Watanabe, M. Itoh, and T.
Hashimoto, “Compact and Low-loss RGB Coupler Using Modeconversion Waveguides,” Opt. Commun., Vol. 420, No. 1, pp. 46–51,
2018.
J. Sakamoto and T. Hashimoto, “Recent Progress in Applications of
Optical Multimode Devices Using Planar Lightwave Circuits,” NTT
Technical Review, Vol. 17, No. 5, pp. 40–44, 2019.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201905ra1.html
H. Takahashi, “Recent Progress on Planar Lightwave Circuit Technology for Optical Communication,” Proc. of SPIE, Vol. 7633, Network
Architectures, Management, and Applications VII, 2009.
M. Sugawara, M. Suzuki, and N. Miyauchi, “Retinal Imaging Laser
Eyewear with Focus Free and Augmented Reality,” Proc. of IDW/AD’16, INP1-2, Fukuoka, Japan, Dec. 2016.

35

Feature Articles

Toshikazu HashimotoGroup Leader, Optoelectronics Integration
Research Group, NTT Device Technology Laboratories.
He received a B.S. and M.S. in physics from
Hokkaido University in 1991 and 1993. He
joined NTT Photonics Laboratories in 1993 andhas been researching hybrid integration of semiconductor lasers and photodiodes on silica-based
PLCs and conducting theoretical research andprimary experiments on the wavefront-matching
method. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) and the Physical Society of Japan.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Junji Sakamoto

Researcher, Optoelectronics Integration
Research Group, NTT Device Technology Laboratories.
He received a B.S. and M.S. in applied physics
from Osaka Prefecture University in 2010 and2012. He joined NTT Device Technology Laboratories in 2012, where he has been researchingsilica-based PLCs. He is a member of IEICE.

36

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Fine-grained Hand-posture
Recognition for Natural
User-interface Technologies
Yuki Kubo
Abstract

NTT Service Evolution Laboratories is conducting research on user-interface technologies to improvesystem usability. As part of this research, we are focusing on new input methods that will be an importantelement of next-generation information devices, and in particular, we are researching hand-posture recognition technology to be used to implement input for glasses-based devices. This article describes initiatives related to this research.
Keywords: user interfaces, sensing technologies, hand-posture recognition

1. Introduction
Information devices are used to display information
to users in all types of situations and have become
indispensable in daily life. Most of these devices areeither displays installed in public spaces or portablemobile devices such as smartphones. Many companies are also researching a new type of wearable
information device that users will wear like glasses,
referred to as smart glasses or augmented reality
glasses. In contrast to previous devices, these glassestype devices have characteristics that display information layered directly over the user’s field of view.
Displaying information directly over the user’s
field of view enables this type of device to layer all
types of information over the real world, presenting
different information to each user. Information in
public spaces, such as train stations and parks, hasconventionally been presented through information
displays that exist physically, such as signage
installed in various locations in a train station to give
directions, displaying the same information to everyone. However, with the ability of glasses-type devices
to display different information to each user, all typesof information that previously had a fixed existencein the real world can change dynamically to a form
suitable for each user. The information each user
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

needs can be displayed to him/her, and the amount of
information displayed can also be adjusted for the
comfort of each user.
We are now connected to information more thanever before, so with information layered over the realworld through glasses-type devices, it will be increasingly important to have an input method for suchinformation, similar to how we input on a smartphone
screen. This article introduces a technology for recognizing fine-grained hand postures, which is currently being investigated and will be necessary fordeveloping natural user interfaces.
2. Related work
One input method that is promising for glasses-typedevices involves gestures, i.e., using the state or
motion of the hands. To make gesture input possible,
the hand gesture has to first be recognized using acamera [1]. However, there are issues with using
cameras: limited range where recognition is possible
due to camera position and field-of-view and recognition difficulty if anything obstructs the camera’s view(i.e., occlusion problem).
How hand gestures will be recognized is important,
but the design of gestures used to perform input isalso important. One can imagine using large arm
37

Feature Articles

movements up, down, left, and right, and having thearms raised for long periods of time as types of hand
gestures, but there are several problems with suchgestures. Large movements or holding the arms up
continuously will increase fatigue and place anincreasing burden on the user over time. Large arm or
body movements also have disadvantages in terms ofsocial acceptance, such as being distracting to people
nearby, being difficult when people or other obstacles
are in close proximity, and allowing people nearby tosee the gestures, thus allowing them to determine
what the user is doing. As one solution for these problems, NTT Service Evolution Laboratories is focusing on input methods that use smaller hand gestures
to reduce the burden on users and is researching afine-grained hand-posture recognition technologythat uses sensors attached to the back of the hand.
Next, we introduce this technology, which is called
AudioTouch [2], based on active acoustic sensing.
3. Technology overview
AudioTouch uses ultrasound waves through principles such as ultrasonography. It detects changes inthe structure (e.g., bones, muscle, and sinew) in theback of the hand and uses them to estimate hand postures. Objects generally have their own resonanceproperties, which depend on features such as shape,material, and boundary conditions. A previous study[3] focused on how the resonance properties of an
object change due to changes in its boundary conditions when the user touches it. They proposed amethod for making touch interfaces from objects
with sensors attached that recognize when the user
touches the object. In contrast, AudioTouch focuses
on changes in resonance properties due to changes inthe shape of the hand itself due to movement of fingers and hands, including the internal structure of thehand. When the user changes his/her hand and finger
postures, the resonance properties of the hand change
due to the movements of parts of the hand, such as
skin, bones, and muscles. By measuring the differences in these properties, the hand and finger postures
can be recognized. Changes in resonance properties
can be measured by sending oscillations (ultrasoundwaves) through the object and measuring the frequency response. AudioTouch observes the frequency response of the back of the hand on the basis of thisconcept to recognize changes in hand and finger
postures. Specifically, it uses two piezoelectric sensors attached to the back of the hand (Fig. 1). The firstemits ultrasonic waves both on the surface and inside
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

LaptopAudiointerface
Piezoelectric sensors

Fig. 1. AudioTouch components.

the hand, and the other measures the ultrasonic waves
propagated over the surface and in the interior of thehand. The captured sound waves are analyzed toobtain the frequency response, and this is used togenerate features, which in turn are used withmachine learning to recognize hand and finger postures.
We conducted experiments to evaluate whether
AudioTouch could recognize three gesture sets
(Fig. 2). The three gesture sets were: one in which the
thumb touches each of the finger joints, one in which
the thumb is moved left, right, up, and down, and one
in which a finger on one hand is used to touch thepalm of the other. We also explored whether levels ofpressure applied when pressing the thumb and otherfingers together could be differentiated into two levels (i.e., strong and weak). Example applications forthese gestures include using the palm as a touchsensitive number keypad or for selecting menu items.
4. Conclusion
In this article, technologies for performing input
with glasses-type devices were discussed and current
initiatives in developing technology for recognizing
fine-grained hand posture, as an element for suchtechnologies, were presented. There are still technicaland social issues to be resolved, and NTT Service
Evolution Laboratories will continue to advance thisresearch.
This work is partially supported by Japan Science
Technology Agency (JST) Advanced Information andCommunication Technology for Innovation (ACT-I)
Project (JPMJPR16UA).

38

Feature Articles

FingerIndex

MiddleThumbs Up
Ring

Thumbs Push

Pinky

A-2

A-3

A-4

A-5

A-6

A-7

A-8

B-1

B-2

B-3

B-4

B-5

B-6

Left

MiddleFinger joints

Top

A-1

Middle
C-1

A-10

A-11

Left

C-2

C-3

Middle

Right

C-4

C-5

N

A-12

Bottom

A-9

Right

Top

A-1
A-5
A-9

20k

A-2
A-6
A-10

A-3
A-7
A-11

A-4
A-8
A-12 40k

Left

Center

B-1

B-2

B-3

B-4

B-5

B-6

40k

20k

Right

Bottom

C-1

20k

C-2

C-3

Neutral

C-4

C-5

N

40k

* Waveforms are examples of waveform-data averages for a given user and label.

Fig. 2. Examples of gesture sets and gesture waveforms.

References
[1]

[2]

D. Kim, O. Hilliges, S. Izadi, A. D. Butler, J. Chen, I. Oikonomidis,
and P. Olivier, “Digits: Freehand 3D Interactions Anywhere Using a
Wrist-worn Gloveless Sensor,” Proc. of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST 2012), pp.
167–176, Cambridge, MA, USA, Oct. 2012.
Y. Kubo, Y. Koguchi, B. Shizuki, S. Takahashi, and O. Hilliges“AudioTouch: Minimally Invasive Sensing of Micro-gestures via[3]

Active Bio-acoustic Sensing,” Proc. of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2019), p. 13, Taipei, Taiwan, Oct. 2019.
M. Ono, B. Shizuki, and J. Tanaka, “Touch & Activate: Adding Interactivity to Existing Objects using Active Acoustic Sensing,” Proc. ofthe 26th Annual ACM Symposium on User Interface Software andTechnology (UIST 2013), pp. 31–40, St Andrews, Scotland, UK, Oct.
2013.

Yuki Kubo

Cybernetic Intelligence Research Project, NTT
Service Evolution Laboratories.
He received an M.E. in systems and information engineering from University of Tsukuba in
2018. He joined NTT Service Evolution Laboratories in 2018. His current research interests
include human-computer interaction and input
techniques. He is a member of the Association
for Computing Machinery, the Virtual Reality
Society of Japan, and Information Processing
Society of Japan.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

39

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Information-display Method for
Reducing Annoyance by Gaze
Guidance
Ryohei Saijo, Tae Sato, Shin-ichiro Eitoku,
and Masahiro Watanabe
Abstract

We are investigating methods of displaying action information to users of smart glasses and otherwearable information devices that are worn daily as they try to attain goals during daily activities. Forcefully presenting information to a user may annoy him/her and discourage him/her from continuing to use
the device, but access to information is uncertain if not forced. To solve this problem, we devised aforced-information display method that reduces annoyance yet ensures that information is received bygiving the user the feeling that the act of reading was him/her choice. We also evaluated the possible
effectiveness of the method.
Keywords: information presentation, gaze guidance, peripheral vision

1. Information display that interrupts
daily activities
Research and development on wearable information devices has been active. Users can benefit invarious ways by receiving information from a display
system that presents timely information via suchdevices. Consider an example in which a device displays “Let’s take the stairs.” when a user that has notbeen doing enough exercise arrives at a place where
there is an elevator and a staircase. The idea is that theuser can improve a daily habit by immediately noticing the suggestion and following it (Fig. 1). However,
the information is useless if it is displayed when theappropriate time has passed (when the user has
already entered the elevator, for example). It is therefore important that the user accesses the information
with certainty when it is displayed. However, if theuser’s attention is obtained forcefully to achieve thatcertainty, the user may regard the display as interrupting him/her current activity. Such feelings might cause
the user to stop using the information-display system.
In scenarios for improving daily habits, the inforNTT Technical Review

Vol. 19 No. 3 Mar. 2021

mation-display system can be used as a tool for user
self-improvement toward personal goals. To support
daily improvement, the system must continuously
and effectively display information to the user. Our
research therefore has the objective of implementing
an information-display method that creates a low
degree of annoyance for the user when information isdisplayed yet ensures a high certainty of information
access [1].
2. Current technology
Much research has been conducted on methods fordisplaying interrupt information. For example, if theuser ignores a prompt to read information, a moreforceful prompt is displayed, and the information is
finally superimposed on the work screen if notaccessed [2]. This approach increases the certainty of
the information being accessed but increases theuser’s feeling of annoyance. One way to reduce theinterruption effect is to estimate the degree of rejecting interrupt information from the device operation
history and using the data to display information at40

Feature Articles

Stairs

Let’s use theelevator.

It’s better totake the stairs.

Visual field while wearing smart glasses

Visual field while wearing smart glasses

Fig. 1. Example of interrupt information display when using smart glasses.

(1) Receive notification(a)(2) Decide(3) Act(4) AccessHigher certainty andhigher feeling ofannoyance

No decision or action

JUMP

Touch
JUMP

(b)
Decide to access and take action
to do so(c)

Omit thedecision

Ignoring
or postponing access
Reduced feeling ofannoyance andreduced certainty of accessInduceactionTouch
JUMPFig. 2. Information-access process.

times when the degree of rejection is low [3]. While
this method reduces disruption of the current task, it
does not allow timely presentation of information. It
is thus difficult to both reduce the feeling of annoyance and increase the certainty of information being
accessed. Although previous research evaluated interference with the user’s work, how the user feels about
the distraction has not been studied. The feeling ofannoyance may be high, even if there is no actualinterruption to the original task. We must therefore
consider the user’s feeling of annoyance.
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

3. Process leading to information access
The process that results in reading information has
four steps: 1) receive notification, 2) decide to read,3) take action to read, and 4) read (Fig. 2). We
focused on steps 2 and 3: decide and take action, andconsidered how the presence or absence of those
steps affect a user’s feeling of annoyance and thecertainty of access.
Consider the process illustrated in Fig. 2(a) to be adisplay method that prioritizes the certainty of access.
41

Feature Articles

Peripheral vision

Change inbrightness
displayedInforEffective visual field

ma

tio

n

I have to look.

Fig. 3. Pseudo-choice method with gaze guidance.

For example, the information is displayed at the location of the user’s gaze, so the user does not decide toattend to the information and does not have to takeaction to see the information. This method is considered to result in a high certainty of access, but it may
produce a greater feeling of annoyance. The methodin Fig. 2(b), on the other hand, can prioritize reduction of the feeling of annoyance. With this method,
the user decides to see the information and takes
action to see it. For example, the user first sees aprompt then touches an icon to view the information.
This method may reduce the feeling of annoyance but
reduces the certainty of access because the user often
postpones access at the decision step.
4. Pseudo-choice interrupt display method
We hypothesized that the certainty of information
access can be improved by omitting the decision step
while reducing the feeling of annoyance of information access by inducing the action step. Based on thishypothesis, we propose a pseudo-choice method inwhich the decision is omitted and the action to accessis induced by the information-display system(Fig. 2(c)).
As one specific implementation of the pseudochoice method, we investigated a gaze-guidancetechnique that uses the change in brightness of theperipheral visual field, which is sensitive to brightness and movement (Fig. 3). This technique guides
the user’s gaze by presenting a brightness change inthe visual periphery to induce the action step without
the decision step.
5. Evaluation of the proposed method
Adapting a dual-task constructed from a main task
and interrupt task, we evaluated the effectiveness of
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

the pseudo-choice method by comparing three different interrupt-task display methods that correspond tothe methods in Fig. 2 (Fig. 4). The experiments first
focused on evaluating the reduction in the feeling ofannoyance. The main task corresponds to daily activities such as watching videos, and the interrupt taskcorresponds to displayed suggestions such as “Let’s
get up for a while.” In daily life, we repeatedly alternate between concentrating on one activity then moving on to another activity. To reproduce such a situation in the experiments, we used two-digit addition
problems to serve as the main tasks and repeatedly
alternated between the situation of solving the problem (activity) and moving on to the next problem(break in activity). To simulate user accessing details
of interrupt information, we used the act of multiplying a two-digit number by a one-digit number to serveas the interrupt task.
The participants were 10 males and females; age
ranging from 20s to 50s. The participants were
instructed to solve as many problems as possible as
they are presented sequentially to encourage concentration on the main tasks. They were instructed to also
solve the interrupt task problems as they appeared.
After the experiments, the participants filled out aquestionnaire asking about their feeling of annoyancefor each experimental condition on a five-point scale.
Statistical analysis of the evaluation results indicatesthat the feeling of annoyance was greater for the nochoice and pseudo-choice conditions than for the
choice condition.
The average scores from the questionnaire (Fig. 5)
indicate that the no-choice and pseudo-choice conditions produced greater perceived annoyance than did
the choice condition, and that the no-choice condition
produced greater perceived annoyance than did thepseudo-choice condition.
These results indicate that the pseudo-choice42

Feature Articles

Main display(2) Display blinking

Secondary display

Interrupt
task

Main task
No-choicecondition(1) Main task

(3) Performinginterrupt task

Keyboard(2) Display blinking

Main task

Choicecondition

pt
rru
te k
In tasMain task(1) Main task
(3) Press thespace key(4) Performinginterrupt task

Main task

pt
rru
te k
In tasPseudo-choice(1) Main task
condition(proposed method)

Main task(2) Display
blinking(3) Performinginterrupt task(a) Experimental conditions and schematic diagrams(b) Experimental setup

Fig. 4. Experimental conditions and setup.

(*: P < 0.05)(**: P < 0.01)

Annoyance*

Feeling of annoyance

5

**

**

4

3

2

1
No annoyance

No choice

ChoicePseudo-choiceFig. 5. Average scores from questionnaire on degree of annoyance.

condition can reduce the user’s feeling of annoyance
relative to the no-choice condition, suggesting thatour pseudo-choice method has the potential to reduce
the feeling of annoyance from the displaying of interrupt information. The choice condition resulted in theleast feeling of annoyance, confirming the potentialof methods that take into account user decision and
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

action to reduce the feeling of annoyance.
6. Future development
In this research, we focused on the movement of theuser’s line of sight as voluntary action for reading
information and confirmed that inducing a gaze43

Feature Articles

movement may result in reducing annoyance of displaying information that interrupts the current task.
The objective of future work is a detailed examination of factors that reduce the feeling of annoyanceand implement a system for displaying information to
improve the every-day behavior of users.

[2]

[3]

References[1]

R. Saijo, T. Sato, S. Eitoku, and M. Watanabe, “Study of Information

Ryohei Saijo

Display Method Focusing on Visual Attention Guidance for Selfimprovement,” IEICE Technical Report, Vol. 119, No. 477, pp. 19–24,
2020 (in Japanese).
R. Yoshida, K. Takahashi, T. Kawamura, and K. Sugahara, “Input
Urging System Using Unpleasant Notification Based on Negative
Motivation,” Proc. of the IEEE Second International Conference on
Electrical, Computer and Communication Technologies (ICECCT2017), pp. 1–5, Coimbatore, India, Feb. 2017.
T. Tanaka and K. Fujita, “Support for Acceptable Interaction Start
Using Ambient Information Presentation Based on User Uninterruptibility Estimation,” Journal of Japan Society for Fuzzy Theory andIntelligent Informatics, Vol. 24, No. 5, pp. 921–932, 2012 (in Japanese).

Shin-ichiro Eitoku

Researcher, Cybernetic Intelligence Research
Project, NTT Service Evolution Laboratories.
He received an M.E. in engineering from University of Tsukuba, Ibaraki, in 2018 and joined
NTT the same year. His research interests
include human-computer interaction and information presentation methods.

Senior Research Engineer, Cybernetic Intelligence Research Project, NTT Service Evolution
Laboratories.
He joined NTT in 2006 and is currently
engaged in researching information systems andmultimedia systems for communications.

Tae Sato

Masahiro Watanabe

Research Engineer, Cybernetic Intelligence
Research Project, NTT Service Evolution Laboratories.
She received an M.E. in engineering from
Tokyo Institute of Technology in 2010 and joined
NTT the same year. Her research interests
include psychology and human-computer interaction.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Senior Research Engineer, Innovative Service
Research Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. in mechanicalengineering and a Dr.E. in electronics, information, and communication engineering fromWaseda University, Tokyo, in 1991, 1993, and2003. He joined NTT Basic Research Laboratories in 1993 and studied the human auditory
system. Since moving to NTT Cyber Solutions
Laboratories in 1999, he has been studying
human-computer interaction, especially universal design and user-centered design.

44

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Presenting Material Properties with
Mid-air Pseudo-haptics
Takahiro Kawabe
Abstract

With the development of image presentation and behavioral-data-acquisition technologies, interaction
between a virtual object and user has become commonplace. This article introduces mid-air pseudohaptics, a technology to provide haptic perception of material properties to the user when the virtual
object is manipulated by the user’s actions. With this technology, the virtual object is deformed by theuser’s actions, and the impression of the object’s softness is controlled by its deformation pattern. In the
future, we will develop technology to provide richer perceptual material properties to virtual objects.
Keywords: mid-air pseudo-haptics, material perception, illusion

1. Motivation

2. Pseudo-haptics

In Hollywood science fiction movies, we often see
scenes in which the main character in a futuristic
world skillfully manipulates images in the air. Whatwould be the haptic nature of images floating in the
air? This simple question is the motivation for thisstudy. Mid-air image representation is no longer science fiction but is now possible with optical devices
and head-mounted displays. NTT’s “Kirari!” immersive telepresence technology is another means ofdisplaying images in the air. Research on presenting
haptic sensation to aerial images (virtual objects) is
being conducted. Various methods have been proposed, but most use devices that provide physicalhaptic sensations.
In this study, we investigated the possibility ofmanipulating the material properties of virtual
objects by changing their appearance in response touser actions. The user obviously cannot directly
touch a virtual object; therefore, we thought that it
would be possible to provide haptic properties to a
virtual object by using perceptual illusion. Specifically, we investigated whether it would be possible toprovide an illusory haptic sensation to a virtual object
when we see a causal relationship between the user’s
manipulation of the object and change in the object’s
appearance.

Before explaining our research in detail, let us consider the pseudo-haptic effect. Suppose a user moves
the cursor displayed on the computer screen with themouse. Now, suppose the user keeps moving themouse in the same manner, but the cursor suddenly
slows down. Studies have shown that this type ofslowdown leaves the user with a sense of resistanceand increases the perceived weight of the cursor. Thiseffect is called pseudo-haptics [1]. Pseudo-haptics is
an illusion that involves multiple sensory modes
including vision, touch, perception, and proprioception, but the effect of vision on other modalities isparticularly critical. Pseudo-haptics can manipulate
impressions of the shape and stiffness of virtual
objects, as well as their weight and resistance.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

3. Mid-air pseudo-haptics
The pseudo-haptic sensation generated when theuser touches a haptic presentation device has been
investigated [2]. In our study, we investigated whether it would be possible to generate pseudo-haptic
sensation when there is no physical contact. We callsuch an effect mid-air pseudo-haptics [3]. To achieve
this effect, two problems had to be solved. The firstwas how to track the user’s hand movements, which
45

Feature Articles

Virtual object

Time

Optical hand-racking sensor

(a)

(b)

Fig. 1. (a) Experimental setup. Hand movement is tracked using an optical hand-tracking sensor, and the virtual object isdeformed in accordance with the movement. (b) Deformation of the object resulting from hand movement.

was easily solved using an optical hand-tracking sensor (Fig. 1(a)). The second problem was what kind ofchanges in the virtual object associated with the
tracked hand motion would create the perception ofmaterial properties. This problem was not straightforward because it involves human perceptual characteristics but is a very interesting question.
4. Key points of this study
We addressed mid-air pseudo-haptics by focusing
on the following three points.
4.1 Poisson effect and Poisson ratio
We investigated the user’s action of pulling a virtual object. When a user applies tension by pulling anelastic object in the horizontal direction, a compressive force acts on it in the vertical direction. For
example, when we pull on a rubber object, it stretches
in the direction of the pull but decreases in thickness
at the center. This phenomenon is called the Poisson
effect. We devised a system in which a virtual object
stretches horizontally according to the movement of
the hand when the user makes a pulling action in thatNTT Technical Review

Vol. 19 No. 3 Mar. 2021

direction (Fig. 1(b)). At the same time, the object
deforms vertically according to the Poisson’s ratio,
which is the ratio of the strain in the material in the
direction of the applied force to the strain occurringin the direction perpendicular to that direction. In thenatural world, the Poisson’s ratio generally does notexceed 0.5. We investigated how the Poisson effectaffected the perception of the stiffness of a virtual
object by varying the Poisson ratio applied to theobject’s deformation.
4.2 R atio of hand movement to object deformation
Previous research [4] on pseudo-haptics has shownthat the smaller the ratio of the amount of change on
the screen to the amount of action by the user, themore resistance and heaviness is perceived. In ourstudy, we varied the ratio of the amount of deformation of the virtual object to the amount of action takenby the user to pull the virtual object and investigated
the relationship between this ratio and perceived stiffness of the virtual object. When this ratio becomeslarger, the amount of deformation of the virtual object
becomes larger for the same amount of movement.
46

Feature Articles

Stiffness of virtual object

0.25

0

Poisson’s ratio
5

5

4

4

3

3

2

2

1

1
1

2

3

4

0.5

1

: p < .05

2

3

4

Ratio of deformation to action (a.u.)
Consistent condition

Inconsistent condition

Fig. 2. Experimental results.

Therefore, the perceived softness of the object should
increase accordingly.
4.3 C onsistency between action and expected
deformation
When a user pulls a virtual object horizontally, andthe object deforms horizontally in response, the relationship between action and deformation is consistent. However, it is not clear whether the consistencyof action and object deformation is necessary to
manipulate the perceived material properties of an
object. We addressed this issue by investigating
experimental conditions for manipulating the consistency of action and object deformation, i.e., consistent and inconsistent. Under the consistent condition,
when the user pulled the object horizontally, theobject extended horizontally. Under the inconsistentcondition, when the user pulled the object horizontally, the object contracted vertically.

than when it had lower Poisson’s ratios. Regarding
the ratio of deformation to action, the higher the ratio,
the less stiff the object was reported as being. It isinteresting to note that regardless of the consistency
between action and deformation, the same pattern ofevaluation was observed.
From these results, the following conclusions canbe drawn.
•	By manipulating the relationship between theuser’s action and deformation of a virtual object,
we can create a mid-air pseudo-haptic sensation
for the user.
•	Using the Poisson effect, it is possible to manipulate the perceived stiffness of a virtual object bychanging the amount of deformation of theobject in accordance with the user’s action.
•	Even if the user’s motion and deformation of thevirtual object do not match, it does not affect the
impression of the object’s softness.

5. Experimental results and discussion

6. The future targeted with this technology

Figure 2 shows the results of an experiment inwhich the stiffness of a virtual object was evaluatedon five levels of Poisson’s ratio. When the object hada Poisson’s ratio of 0.5, it was judged to be less stiff

The Innovative Optical and Wireless Network(IOWN) will achieve high-speed, high-capacity communications in 2030. At that time, technology to present a large amount of information to users in a natural

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

47

Feature Articles

manner will be essential. As described in the IOWN
concept document, NTT is aiming for a futuristic userinterface “Point of Atmosphere (PoA)” to create aworld where people are in harmony with their digital
environment without being aware of surrounding
devices [5].
The pseudo-haptic technology proposed in thisstudy enhances the sense of realism in the virtualworld by extending the experience of real object
manipulation to a virtual object so that the virtual
object appears to stretch when the user makes a pulling action. This type of object manipulation andimpression of material properties in augmented reality will be necessary for future interface design. We
hope to bring PoA closer to reality by promoting
basic research for the development of not only midair pseudo-haptics but also perception-based presentation technology.

References
[1]

[2]

[3]

[4]

[5]

A. Lécuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet,“Pseudo-haptic Feedback: Can Isometric Input Devices Simulate
Force Feedback?”, Proc. of IEEE Virtual Reality 2000 (Cat.
No.00CB37048), pp. 83–90, New Brunswick, NJ, USA, Mar. 2000.
https://doi.org/10.1109/vr.2000.840369
A. Lécuyer, “Simulating Haptic Feedback Using Vision: A Survey of
Research and Applications of Pseudo-haptic Feedback,” Presence:
Teleoperators and Virtual Environments, Vol. 18, No. 1, pp. 39–53,
Feb. 2009.
https://doi.org/10.1162/pres.18.1.39
T. Kawabe, “Mid-air Action Contributes to Pseudo-haptic Stiffness
Effects,” IEEE Transactions on Haptics, Vol. 13, No. 1, pp. 18–24,
Dec. 2019.
https://doi.org/10.1109/toh.2019.2961883
A. Lécuyer, J.-M. Burkhardt, and L. Etienne, “Feeling Bumps andHoles without a Haptic Interface: the Perception of Pseudo-haptic
Textures,” Proc. of the SIGCHI Conference on Human Factors in
Computing Systems (CHI ’04), pp. 239–246, Vienna, Austria, Apr.
2004.
https://doi.org/10.1145/985692.985723
J. Sawada (supervisor), M. Ii, and K. Kawazoe (authors), “IOWN:
Beyond the Internet,” NTT Publishing, 2019 (in Japanese).

Takahiro Kawabe

Senior Distinguished Researcher, Leader of
Sensory Representation Group, Human Information Science Laboratories, NTT Communication
Science Laboratories.
He received a Doctor of Psychology from
Kyushu University, Fukuoka, in 2005. In 2011,
he joined NTT Communication Science Laboratories, where he studies applied aspects of humanperception. He received the 2013 JPA Award forInternational Contributions to Psychology:
Award for Distinguished Early and Middle
Career Contributions from the Japanese Psychological Association. In 2018, he was also awarded the Young Scientists’ Prize of the Commendation for Science and Technology by the Ministerof Education, Culture, Sports, Science andTechnology. He is a member of the Vision Sciences Society and the Vision Society of Japan.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

48

Feature Articles: Media Robotics as the Boundary
Connecting Real Space and Cyberspace

Evaluation of Adaptability to
Unfamiliar Environments Using
Virtual Reality
Takashi Isezaki and Tomoki Watanabe
Abstract

To prevent accidents when the elderly walk or drive a car, it is important to improve their motor functions, but their cognitive ability to adapt to changes in their environment so that they can move appropriately for their activity is also important. We are conducting research to develop technology for evaluating
and improving the ability to adapt to the environment while moving. This article introduces initiatives
toward such technologies.
Keywords: VR, cognitive function, manual dexterity

1. Significance of evaluating ability to adapt tothe environment while moving
Accidents related to mobility for the elderly, suchas falling down and automobile accidents, are societal issues. Falling while walking accounts for a significant proportion of deaths among the elderly.
Elderly persons also account for 18.1% of traffic
accidents, and the proportion of accidents caused byelderly drivers tends to increase yearly [1].
NTT Service Evolution Laboratories is conducting
research on estimating users’ mobility function frominertial data and surface electromyogram data [2, 3].
Mobility behaviors, such as walking or driving a car,are considered to be performed based on a generalmotor control model. This involves a sequence ofprocesses: that of perceiving the state of one’s bodyand the external environment through visual, auditory, and somatic senses; cognitive processing of theperceived information in the brain; sending instructions to the musculoskeletal system; and the mobility
processes in which the musculoskeletal system
receives the instructions from the brain and muscles
contract or relax accordingly. To understand thecauses of elderly people falling down while walking
or being involved in traffic accidents and prevent such
incidents, it is necessary to examine all of these faculNTT Technical Review

Vol. 19 No. 3 Mar. 2021

ties, not just mobility. Cattell subdivided intelligence
into two aspects: crystallized intelligence and fluid
intelligence [4]. Fluid intelligence is the ability tolearn new things or solve problems to adapt to newenvironments, and is thought to decline with age.
Conversely, crystallized intelligence includes theabilities an individual acquires through experience,
training, and practice throughout their life and tends
to be stable despite the passing of years. When walking or driving a car, one often leaves familiar roads
and drives on new and unfamiliar routes or encounters unexpected situations such as pedestrians who
suddenly appear. When appropriate mobility behaviors are required in an environment that is changing
from minute to minute, fluid intelligence, or thecognitive ability to adapt immediately to unfamiliar
or unexpected aspects in the environment, is important. Therefore, evaluating and improving the user’s
ability to adapt to unfamiliar or unexpected environments while moving are important in preventing
accidents related to mobility.
2. Virtual-reality system for evaluating ability toadapt to the environment while moving
To measure users’ ability to adapt to their environment
requires observation of their behavior in unfamiliar
49

Feature Articles

and unexpected environments, evaluation and analysis of the environment, and their bodily movement inresponse. Various manufacturers have recently begun
selling consumer-oriented virtual reality (VR) systems that anyone can use. VR is able to simulate andreproduce various environments and provide information on those environments and the user’s movements in those environments. The concept for thisresearch was to use VR to create environments thatare unfamiliar to the user then measure their ability toadapt to those environments by observing their movements and performance in the environments.
To evaluate the ability to adapt to the environment
while moving requires analysis of bodily information, including how behaviors differ when the ability
to adapt to the environment is strong or weak. The
level of dexterity (skill) with the upper limbs (including upper arms, forearms, and hands) is understood tobe related to cognitive functions [5]. Considering theability to adapt to the environment while moving as
one of these cognitive functions, our concept was toevaluate this ability by focusing on dexterity in theupper limbs. There are many methods for evaluating
dexterity in the upper limbs, but three basic upperlimb mobility functions have been identified related
to dexterity, which are spacing, timing, and grading[6]. Spacing is the ability to move the hand in the correct direction, timing is the ability to move the handby correctly adjusting the timing, and grading is theability to correctly adjust the effort applied whilemoving. NTT Service Evolution Laboratories is conducting joint research with Prof. Kensaku Mori and
Assoc. Prof. Masahiro Oda of the School of Information Science, Nagoya University, and Prof. Hitoshi
Hirata and Designated Assistant Prof. Shintaro
Oyama of the Department of Hand Surgery in the
School of Medicine, Nagoya University, for implementing a system using VR to evaluate the ability toadapt to the environment while moving that is basedon the dexterity of the upper limbs.
For upper limb dexterity, we focused specificallyon the task of catching a falling ball. To simulate anenvironment in which the users were not accustomed
to, we conceived three scenarios that would produce
differences from behavior in the real world. The firstwas to change the characteristics of the object (ball).
In particular, we examined changes in the acceleration of gravity and the size of the object and createdan implementation in which these parameters arevariable. The second was to change the characteristics of the environment space. We examined spaces
with obstacles that cause the object to bounce and in
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

which the acceleration of the object (ball) changes
three dimensionally. We implemented these obstacles
and the space so that the range, rebound coefficients,
and acceleration settings could be set arbitrarily. The
third scenario was to change the motion characteristics of the user. In this case, we set the parameters,i.e., coefficients used to project real-world upperlimb movements onto movements of the avatar in VR
space; either exaggerating or attenuating the movements.
Figure 1 shows images of using a prototype of ourVR system. In this task of catching a falling ball fromthe scenario of changing the characteristics of theenvironment space, a prescribed range (shown in red)is an obstacle, which bounds the environment inwhich the ball is falling, simulating changes in thecharacteristics of the environment. Through multiple
trials, the users were able to adapt to this unfamiliar
environment and consistently catch the ball. The system measures the user’s movement data (from thecontroller) and their performance while performing
the task in the specified environment. A sequence ofuser operations for one task (Fig. 1(a)) and the user’s
VR perspective (Fig. 1(b)) are shown. As shown in
Fig. 1(a), the user in the real world wears a headmounted display (HMD) and uses controllers, which
are part of the VR system. The video shown on thedisplay in Fig. 1(a) is the VR data that the user
receives through the HMD. Video selected from what
the user sees in the HMD is shown in Fig. 1(b). The
left is a scene with the red ball falling from above, thecenter shows the ball bouncing off of the obstacle (thered region), and the right shows the ball having beencaught by the user.
Figure 2 shows graphs of (a) the cumulative number of successes and (b) reproducibility of the motion
when a single user performed multiple trials of thetask. Reproducibility of motion is computed as thecorrelation coefficient for the motion data between
trials. It is clear that both the cumulative successes
and movement reproducibility increased with thenumber of trials. By quantifying changes in performance such as success rates and motion reproducibility as trials accumulate, we expect to be able toevaluate the user’s ability to adapt to the environment
while moving.
3. Future development
This article introduced a VR system for evaluating
the ability to adapt to an environment while moving. It
enables movement data and performance measurements
50

Feature Articles(a) Experimental setup for task of a user trying to catch a falling ball(b) Video from the user’s perspective

12
10

Correlation coefficient

Cumulative number of successes

Fig. 1. Images of a prototype of our VR system.

8
6
4
2

0.8
0.6
0.4
0.2
0.0

0
0

5

10

15
20
25
Number of trials

3035(a) Change in cumulative successes

0

5

10

15
20
25
Number of trials

3035(b) Change in reproducibility of motion

Fig. 2. An analysis example from a user.

to be collected during the process of adapting tochanges in the environment. We intend to continue
studying methods for evaluating the cognitive function of the ability to adapt to the environment whilemoving, on the basis of data from participants withvarious attributes, using measured motion and performance data. We also intend to continue updating thetasks and configurations in our prototype VR system.
We plan to investigate the significance of this evaluation of adaptability in the real world, beyond fallingor traffic accidents, to issues with a wide range ofcognitive functions, such as forgetfulness. We will
also investigate how this system can be used to
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

improve adaptability to the environment while moving.
During the current novel coronavirus pandemic, we
frequently encounter situations in which we cannot
go out for a walk or other exercises. The need fortechnologies for training our bodies while at home ishigher than ever before. Many devices are beingbrought to market that are designed to improve
mobility-related functions, such as muscle mass, andwe expect that training cognitive functions using
devices/systems, such as that introduced in this article, will also become more common. We will contributeto extending the healthy lives of everyone by creating
51

Feature Articles

technologies to evaluate and improve cognitive function in various ways and implement them in peoples’
homes as evaluation and training systems.
References[1][2]

Tokyo Metropolitan Police Department, Preventing Traffic Accidents
Involving the Elderly (in Japanese), https://www.keishicho.metro.
tokyo.jp/kotsu/jikoboshi/koreisha/koreijiko.html
T. Isezaki, N. Fujii, S. Ishii, T Watanabe, and O. Mizuno, “Physical
Function Estimation Based on Inertial Data of the Motion of SteppingOver an Obstacle,” 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society (EMBC 2016), Orlan-

Takashi Isezaki

Researcher, Cybernetic Intelligence Research
Project, NTT Service Evolution Laboratories.
He received a B.E. and M.E. from Universityof Tsukuba in 2011 and 2013 and a Ph.D. inengineering from University of Tsukuba in 2019
He joined NTT Service Evolution Laboratories
in 2013. His research interests include biologicalsignal processing and modeling biological systems.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

[3]

[4]
[5]

[6]

do, FL, USA, Aug. 2016.
T. Isezaki, H. Kadone, A. Niijima, R. Aoki, T. Watanabe, T. Kimura,and K. Suzuki, “Sock-type Wearable Sensor for Estimating Lower
Leg Muscle Activity Using Distal EMG Signals,” Sensors, Vol. 19,
No. 8, 1954, 2019.
R. B. Cattell, “Theory of Fluid and Crystallized Intelligence: A Critical Experiment,” J. Educ. Psychol., Vol. 54, No. 1, pp. 1–22, 1963.
J-Y. Yoon, T. Okura, K. Tsunoda, T. Tsuji, Y. Kohda, Y. Mitsuishi, C.
Hasegawa, and H. Kim, “Relationship between Cognitive Function
and Physical Performance in Older Adults,” Japanese J. Phys. Fit.
Sport. Med., Vol. 59, No. 3, pp. 313–322, 2010.
T. Fukui, K. Inoue, and K. Tunehisa, “Development of an Apparatus
for Hand Dexterity Evaluation: Study of a Clinical Adaptation,”
Kawasaki Medical Welfare Journal, Vol. 17, No. 2, pp. 389–394, 2008
(in Japanese).

Tomoki Watanabe

Senior Research Engineer, Innovative Service
Research Project, NTT Service Evolution Laboratories.
He received a B.E. from Yokohama National
University, Kanagawa, in 1992, and a Ph.D. inengineering from Kanagawa Institute of Technology in 2015. He joined NTT Human Interface
Laboratories in 1992. His research interests
include human-computer interaction and network technology for Internet of Things services.

52

Regular Articles

Increasing Capacity of a Strictly
Non-blocking Clos Network
Composed of Optical Switches
Toru Mano, Takeru Inoue, Kimihiro Mizutani,and Osamu Akashi
Abstract

Clos networks are widely used due to their non-blocking property. However, in strictly non-blocking
Clos networks composed of ordinary optical switches (switches with equal numbers of input/output
ports), we find that a substantial number of ports can remain unused, which decreases their efficiency.
This inefficiency comes from the implicit restriction that the two sides of switches have distinct roles,
i.e., ports on one side are connected to endpoints (terminals) while those on the other side are linked toswitches. Removing this restriction provides greater freedom in constructing a network and can increase
capacity without losing the strictly non-blocking property. We propose a new strictly non-blocking
network and provide several theorems that address network capacity. Numerical experiments show thatthe proposed network has up to 30% more capacity compared with a strictly non-blocking Clos network.
Keywords: non-blocking switching networks, Clos network, network design

1. Introduction
Constructing non-blocking switching networks [1],where connections can be established between any
pair of idle ports, remains a vital issue in the field of
network design. Such networks have been applied inmany areas such as telephone networks [2], asynchronous transfer mode networks [3], optical switching with wavelength-division multiplexing technologies [4], and time-division packet switching [5]. A
rich body of literature has been produced on nonblocking networks with the goal of constructing
larger-capacity networks [6, 7]. Clos networks [2] areclassic networks, and their capacity has been extensively analyzed [8], e.g., 10,000 connections can beaccommodated using 59 2000-port switches arranged
in a three-stage network.
To the best of our knowledge, studies on non-blocking networks made the implicit restriction thatswitches have two facing sides with several ports, andonly one side can be used to connect to endpoints [9,NTT Technical Review

Vol. 19 No. 3 Mar. 2021

10]. For example, in Fig. 2, for switches in the input
layer, the left side is connected to endpoints (t’s)
while the right side is linked to switches. This restriction makes it easy to understand the network, but it
might decrease the efficiency of a Clos network if itconsists of ordinary square switches (switches withequal numbers of input/output ports) [11, 12]. As
discussed in Section 3, a substantial number of ports
has to remain unused in a Clos network, e.g., in Fig.
2, if unused (black) ports are connected, the capacity
would not increase or the network would lose its nonblocking property.
We focus on a non-blocking switching networkwith greater capacity than the Clos equivalents. To
increase capacity, we remove the above implicit
restriction while still using square switches, whichhave equal numbers of input/output ports, as used inprevious studies [9, 10]. To the best of our knowledge, this is the first study on maximizing the capacity of non-blocking networks that use square switches
without the above restriction. Among the several
53

Regular Articles

Fig. 1. All four possible configurations in a square switch of N = 4.

known non-blocking properties [1], we investigated
non-blocking networks in the strict sense (any idle
port can be connected independent of current connections) with the most fundamental form of spacedivision multiplexing*1. Note that we focus on networks with only a single intermediate layer to mitigate signal attenuation. Since our focus is on the strict
non-blocking property, routing issues are not discussed.
The contributions of this article are summarized asfollows.
•	Section 3 reveals that a substantial number ofports can remain unused in Clos networks.
•	Section 4.1 proposes a strictly non-blocking network that is based on a Clos network.
•	Section 4.2 proves that the proposed network has
equal or larger capacity than a Clos network.
•	Numerical evaluations with reasonable parameters showed that the proposed network has up to
about 30% larger capacity than a Clos network(Section 5).
2. Preliminaries
2.1 Problem statement
The basic switching component is represented as asquare switch, as illustrated in Fig. 1. A square switch
has two sides, and each side has N ports. Althoughinput and output ports can coexist on the same side,
we are only allowed to connect an input port on one
side with an output port on the other side. We assume
that all switches have equal N. The size of a squareswitch is defined as N.
A switching network is defined as a set of switches
connected by directional links (optical fibers), andthe network is connected to endpoints, as illustrated
in Fig. 2. Ports connected to an endpoint are called
external ports, while the other ports are called internal ports. Ports without links are called unused ports.
Since links have a direction, all external/internal ports
and endpoints are classified as either transmitting or
receiving. Every transmitting port/endpoint is
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

allowed to connect only with a receiving one. We
consider switching networks with a single layer ofintermediate switches (intermediate switches arethose without external ports) such as one-sided twostage (Fig. 2 and Fig. 3) and two-sided three-stage(Fig. 4) networks.
Connection requests are classified into two categories: creation and termination. A creation request,
specified by a pair of idle endpoints, triggers theestablishment of a path between the endpoints. In
Fig. 2, two endpoints t11 and r21 are specified, and thethick gray line represents the path establishedbetween them. A termination request, which specifies
an existing connection, releases the corresponding
path. Note that the request sequence is unknown inadvance.
Strictly non-blocking networks are designed toaccept any request sequence without requiring therearrangement of existing paths. Moreover, paths
used to satisfy creation requests should be chosen
arbitrarily. Thus, the path-selection algorithm is outside the scope of this article.
Our problem, the connection maximization problem, is defined as follows. Given the number ofswitches, construct a strictly non-blocking networkthat has the maximum number of connections. Thismaximum number of connections is the capacity of
the network. In Fig. 2, given seven switches, eight
connections can be established between the eighttransmitting endpoints and eight receiving endpoints.
2.2 Current approach: Clos networks
This subsection reviews current non-blocking networks, i.e., their connection schemes, non-blocking
conditions, and formulations of the connection maximization problem. We explain two current Clos networks: unfolded Clos [2, 6, 9] and folded Clos [7, 10].
Note that an unfolded Clos network is usually called
just a Clos network, but to eliminate the ambiguity*1 Extension to time-division, wavelength-division, and code-division multiplexing [7] is for our future work.

54Regular Articles

r 11

t 11

r 21

t 21

P1
P5

r 12
t 12
t 22

r 22

P2

P6

r 13
t 13
r 23

P3

t 23

P7

r 14
t 14

P4

r 24
t

4
2

Endpoints

Switches

Fig. 2. A n example of strictly non-blocking switching network (N = 6). External switch ports are gray, internal switch ports
are white, and unused ports are black.

t 11
t n1
r 11

n fibers

P1

r n1

m fibers

Pk + 1

t 12
t n2
r 12

P2

r n2
Pk + l
t 1k
t nk
r 1k
r nk

Pk

Input-output layer(k switches)Intermediate layer(l switches)

Fig. 3. A folded Clos network consisting of k + l switches. This network offers up to n · k connections.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

55

Regular Articles

between folded and unfolded, we explicitly call it an
unfolded Clos network. The disadvantages of these
networks are analyzed in Section 3. These networks
are similar, but they have different capacities, as discussed in Section 5.
As explained in the Introduction, these networks
have an implicit restriction; ports on one side are
either all external or all internal; mixing is not permitted.
2.2.1 Folded Clos networks
Figure 3 illustrates a folded Clos network [7, 10]. A
folded Clos network has k > 2 switches in the inputoutput layer and l switches in the intermediate layer.
The case of k ≤ 2 is ignored because it can support at
most N connections, and a single switch is sufficient.
The right side of each switch in the input-output layer
is connected to each side of the switches in the intermediate layer with m fibers. The left side of eachswitch, Pi, in the input-output layer is connected to ntransmitting endpoints, {tji}j = 1,…, n, and n receiving endpoints, {rji}j = 1,…, n. Hence, a folded Clos
network can serve n · k connections in total. In Fig. 3,
substituting 4, 3, 1, 2 with k, l, m, n, respectively,
yields the network in Fig. 2.
The network is strictly non-blocking if and only if[7]
2 n−1 + 1 ≤ l.
(1)m		
Therefore, as illustrated in Fig. 2, if we increase thenumber of external ports, n, to 3 from 2, i.e., 3 transmitting external ports and 3 receiving ones, then the
network loses its strictly non-blocking property.
In folded Clos networks that use at most a switcheswith N ports, the maximum number of connections,
i.e., the capacity fC(a; N) can be obtained by solving
the following problem over positive integer variablesk(> 2), l, m, n ∈ .
max
s.t.

n ∙ k(2)n ≤ N/2 (3)m ∙ l ≤ N/2 (4)m ∙ k ≤ N(5)2 n−1 + 1 ≤ l(6)mk + l ≤ a.(7)Objective (2) represents the number of connections ina folded Clos network, and Constraints (3), (4), and(5) infer that each side of the switch has enough ports
for fiber provision. Constraint (6) guarantees the nonblocking property, and Constraint (7) restricts the
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

number of switches used. We say the connection
parameters (k, l, m, n) are feasible for a folded Clos
network if they satisfy Constraints (3), (4), (5), and(6).
2.2.2 Unfolded Clos networks
Figure 4 illustrates an unfolded Clos network [2, 6,
9]. Its connection scheme is similar to that of folded
Clos networks. We consider only k > 1 for the same
reason given for folded Clos networks.
This network is strictly non-blocking if and only if
[6]
2 n−1
m

+1 ≤ l.(8)As with a folded Clos network, the maximum number of connections, i.e., the capacity fU (a; N), can beobtained by solving the following problem over positive integer variables k(> 1), l, m, n ∈ ℕ.
max
s.t.

n ∙ k(9)n ≤ N(10)m ∙ l ≤ N(11)m ∙ k ≤ N(12)2 n−1 + 1 ≤ l(13)m
2k + l ≤ a.(14)3. Disadvantages of using Clos networks
This section explains the disadvantages of Clos
networks. Moderately sized strictly non-blocking
folded Clos and unfolded Clos networks must haverelatively large numbers of unused ports.
Proposition 1. Let s be an integer greater than 1. A
strictly non-blocking folded Clos network that canserve at least sN connections has at least 12 1− 1s N
unused ports on the left side of the input-output
switch. That is, if connection parameters (k, l, m, n)
are feasible for this network and satisfy nk ≥ sN, thenN − 2n ≥ 12 1− 1s N.
Proof. Let (k, l, m, n) be feasible connection parameters with nk ≥ sN. We have k ≥ 2s because of Constraint (3) and condition nk ≥ sN. Let q be an integer
such that q = n−1, then we have n ≤ (q + 1)m and 2qm+ 1 ≤ l from Constraint (6). Together with Constraints(4) and (5), we have 2n ≤ (l + 1)m ≤ 2n ≤ (l + 1)m ≤NN+ 2s= 12 1 + 1s N.
2
Proposition 2. Let s be an integer greater than 1. A
strictly non-blocking unfolded Clos network that canserve at least sN connections has at least 12 1− 1s N
unused ports on the left/right side of the input/output
switch. That is, if connection parameters (k, l, m, n)

(

(

(

)

(

)

)

)

56

Regular Articles

t 11

n fibers

n fibers
m fibers

m fibers
Pk + l + 1

P1
t

r 11

1
n

r n1

Pk + 1

r 12

t 12
Pk + l + 2

P2

r n2

t n2

Pk + l
r 1k

t 1k
P2k + l

Pk

r nk

t nk
Input layer(k switches)

Output layer(k switches)Intermediate layer(l switches)

Fig. 4. An unfolded Clos network consisting of 2k + l switches. This network offers up to n · k connections.

are feasible for this network and satisfy nk ≥ sN, thenN − n ≥ 12 1− 12 N.
Proof. Let (k, l, m, n) be feasible connection parameters with nk ≥ sN. We have k ≥ s because of Constraint (10) and condition nk ≥ sN. Let q be an integer
such that q = n−1, then we have n ≤ (q + 1)m andmhave 2q + 1 ≤ l from Constraint (6). Together withN
Constraints (11) and (12), we have n ≤ l+1
m ≤ N2 + 2s21
1
= 2 1 + s N.
The above propositions show that the capacity offolded Clos or unfolded Clos networks increases andmore ports are unused: the number approaches N2
asymptotically. This was also observed through thenumerical evaluation discussed in Section 5.

(

(

)

t 11

n fibers

t n1
r 11

P1

r n1

m fibers

Pk + 1)

4. Proposed network
In this section, we explain the proposed networkthat is based on folded Clos networks. First, we
explain the proposed network and its non-blocking
condition. We then compare its capacity with that ofClos networks from the theoretical viewpoint.
4.1 P
 roposed network and its non-blocking condition
Figure 5 illustrates the proposed network. Bothsides of the input-output layer are connected to endpoints. At the left side of each switch in the inputNTT Technical Review

Vol. 19 No. 3 Mar. 2021

Pk + l
r 1k

Pk

r nk
t 1k
t nk
Input-output layer(k switches)Intermediate layer(l switches)

Fig. 5. T he proposed switching network consisting of k + lswitches. This network offers up to n · k connections.

output layer, n ports are connected to receiving endpoints, while every m port is connected to the down
57

Regular Articles

side of a switch in the intermediate layer. The rightside mirrors this configuration. The proposed network offers n · k connections in total. We consider
only k > 2 as per a folded Clos network. We confirmed that the construction of the proposed network
is feasible on commercially available optical switches.
The strictly non-blocking condition, Condition (1),does not change, and the proof is almost the same as
for a Clos network [2, 6] (the proof is given at the endof this subsection).
Proposition 3. The proposed network is strictly nonblocking if and only if
2 n−1 + 1 ≤ l.
m(15)

The maximum number of connections, i.e., thecapacity fP(a; N), can be obtained by solving the following problem on positive integer variables k(> 2), l,m, n ∈ .
maxs.t.

n ∙ k(16)n + m ∙ l ≤ N(17)m ∙ k ≤ N(18)2 n−1 + 1 ≤ l(19)m
k + l ≤ a.(20)Compared to a folded Clos network, the constraints
related to input-output square switch size, i.e., Constraints (3) and (4), are replaced with Constraint (17).
The other constraints are the same.
As discussed in Section 5, the proposed network
has less unused ports compared with current networks because the proposed network does not havethe implicit restriction, that is, a single side in the
proposed network can have both external and internal
ports.
Finally, we give a proof of Proposition 3.
Proof of Proposition 3. First, we discuss sufficiency.
Without loss of generality, we can assume that aconnection-creation request (t11, r12 ) has arrived.
Because P1 uses at most n−1 fibers connected totransmitting endpoints, at most n−1 fibers that go tointermediate switches from P1 are used. With thesame argument, at most n−1 fibers that come to P2
from intermediate switches are used. Hence, at mostn−1
intermediate switches run out of available fibersmthat come from P1, and at most n−1
intermediatemswitches run out of available fibers that go to P2.
Thus, if we have 2 n−1+1 intermediate switches,mthere exists at least one intermediate switch that has
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

an available fiber coming from P1 while another one
goes to P2. By using this intermediate switch, we canestablish the connection.
Next, we discuss necessity. Suppose l is 2 n−1,m
and connection-creation requests {(ti1, ri2)}i = 1,…,
n–1 and {(ti2, ri3)}i = 1,…, n–1 have arrived; the former requests {(ti1, ri2)} use the first l/2 intermediateswitches, and latter requests {(ti2, ri3)} use the last l/2
intermediate switches. Note that a strictly non-blocking network has to establish a connection between
idle ports regardless of the past path selection. However, connection-creation request (t n1, rn3) cannot
occur because the first l/2 intermediate switches do
not have any available fiber coming from P1 and thelast l/2 intermediate switches do not have any available fiber going to P3.
4.2 Capacity comparison
This subsection compares capacity from a theoretical viewpoint. We show that the proposed network
has equal or larger capacity than a folded Clos network (Theorem 1). With regard to an unfolded Clos
network, we give only a sufficient condition for the
proposed network to have equal or larger capacity(Theorem 2). Due to the complexity of the discrete
nature of integers, it is intractable to analyze theremaining case, i.e., the case that Theorem 2 does notcover. To compare the remaining case, we conducted
a numerical evaluation, as discussed in Section 5.
First, we prove that the capacity of the proposed
network fP(a; N) is greater than or equal to that of afolded Clos network fC(a; N).
Theorem 1. The proposed network has equal or
larger capacity than a folded Clos network. That is,
we havefC(a; N) ≤ fP(a; N)(21)when there is a feasible solution for a folded Clos
network.
Proof. Suppose that the connection parameters (k, l,m, n) are feasible for a folded Clos network. That is,(k, l, m, n) satisfy Constraints (3), (4), (5), (6), and (7).
Then, it is sufficient to show that (k, l, m, n) are also
feasible for the proposed network. The only difference between the proposed and folded Clos networks
is that a folded Clos network has Constraints (3) and(4), whereas the proposed network has Constraint(17). We have N2 + N2 ≤ N. Since Constraints (3) and(4) infer another Constraint (17), (k, l, m, n) are also
feasible for the proposed network.
Next, we give a sufficient condition for the
58

4000
3000
2000
1000
0
0

50

100

150

200

250

300

500,000
400,000
300,000
200,000
100,000
0
0

500

1000

1500

2000

2500

3000

Capacity(number of external ports)

5000

Capacity(number of external ports)

Capacity(number of external ports)

Regular Articles

5 × 107
4 × 107
3 × 107
2 × 107
1 × 107005000 10,000 15,000 20,000 25,000 30,000

Number of switches

Number of switches

Number of switches(a) N = 100(b) N = 1000(c) N = 10,000

Proposed fP
Unfolded fU
Folded fC

Fig. 6. T he capacity and number of switches used. We examined three square switch sizes N = 100 (a), 1000 (b), and 10,000(c). All three cases generated almost identical plots.

proposed network to have equal or larger capacity toan unfolded Clos network.
Theorem 2. If one of the optimal configurations of an
unfolded Clos network has even m, then the proposed
network has equal or larger capacity, i.e., fU(a; N) ≤fP(a; N), where m is the number of connections
between the input/output switches and intermediateswitches.
To prove Theorem 2, we use the following proposition and lemma. First, we give a proof of the lemma,then provide a proof of the proposition, and finallyprove Theorem 2.
Proposition 4. Let (k, l, m, n) be feasible for an
unfolded Clos network and m be an even integer.
There exist feasible parameters (k’, l’, m’, n’) for thefolded Clos network such that nk ≤ n’k’ and 2k + l ≥k’ + l’.
Lemma 1. Let (k, l, m, n) be feasible for an unfolded
Clos network, m be an even integer, and n be an odd
integer. Parameters (k, l, m, n+1) are also feasible forthe unfolded Clos network.
Proof of Lemma 1. Let q be an integer such that q =n−1. It is sufficient to show n+1 ≤ N and mn = q.
m
Since n is odd and m is even, we have qm + 1 ≤ n ≤m(q+1) − 1. This inequality gives us mn = q. In addition, since Constraint (11) is satisfied, we have n+1 ≤m(q+1) ≤ m(2q+1) ≤ ml ≤ N.
Proof of Proposition 4. Due to Lemma 1, we canassume that n is also even. Hence, it is sufficient toshow that (k’, l’, m’, n’) = 2k, 2 n−1+ 1, m2 , n2m
are also feasible for the folded Clos network. That is,
we show that (k’, l’, m’, n’) satisfy Constraints (3),(4), (5), and (6). Since m and n are even, Constraints(3), (4), and (5) are satisfied. Let us consider theremaining constraint, Constraint (6). Let q be an integer such that q = n−1, then by using integer r ∈ {0,m(

NTT Technical Review

Vol. 19 No. 3 Mar. 2021)

1, ..., m−1}, n can be represented as n = mq + r +1. By
dividing by 2 and subtracting 1, we have n’–1 = m’q+ (r+1)/2 − 1. Since m and n are even, the residue(r+1)/2 − 1 is one of {0, 1, ..., m’−1}. Hence, we have
2 (n'−1)+ 1 = 2 n−1+ 1 = l'.
m'm
Proof of Theorem 2. We prove the theorem by contradiction using Theorem 1 and Proposition 4. Supposethat there exists a configuration of an unfolded Clos
network with even m that has a larger capacity thanthe proposed network. Then, by Proposition 4, we canconvert this unfolded Clos network into a folded Clos
network without losing capacity. Hence, the converted folded Clos network has larger capacity than the
proposed network. However, this contradicts Theorem 1.
5. Numerical evaluation
This section numerically compares the proposed
network with folded and unfolded Clos networks*2.
We now show that the proposed network has
improved capacity.
First, we show that switch size slightly affects theresults. Figure 6 shows the capacity (y-axis) andnumber of switches used (x-axis). We plot the results
for the folded Clos, unfolded Clos, and proposed networks, that is, fC(a; N), fU(a; N), and fP(a; N). We used
three switch sizes, N = 100 (Fig. 6(a)), 1000
(Fig. 6(b)), and 10,000 (Fig. 6(c)). All three cases
generated almost identical plots. We confirmed that
*2 The capacity of an unfolded Clos network matches that of a folded Clos network. For example, let the switch size be N =100 andthe number of switches be one of {8, 9, ..., 12}. Then the unfolded Clos network has larger capacity for a = 8, 12, and the folded
Clos network has larger capacity for a = 9, 10, 11.

59

40,000
30,00020,00010,000
0

0

50
100
150
Number of switches

200(a) 0  a  200

Capacity(number of external ports)

Capacity(number of external ports)

Regular Articles

200,000
150,000
100,000
50,000
0
200

400

600
800
1000
Number of switches

1200(b) 200  a  1200

Proposed fP
Unfolded fU
Folded fC

Fig. 7. The capacity and number of switches used for smaller a. These plots were obtained by enlarging Fig. 6(b).

other switch sizes, N ∈ {200, 300, ..., 900} {2000,
3000, ..., 9000}, generated almost identical plots.
These results imply that switch size slightly affects
capacity comparison. Note that, in the folded Clos,
unfolded Clos, and proposed networks, the line
became flat when a exceeded around 3/2N, 3N, and5/3N, respectively. The reason is that capacity does
not increase even if we add more available switches
since we run out of internal ports that the switches
can use to connect with each other. One way to evade
this situation is adding another intermediate layer.
However, as explained in Section 2.1, we considered
networks with a single intermediate layer. Extension
to multistage networks is for future work.
Next, we compared the capacity of the proposed
network with folded Clos and unfolded Clos networks. We investigated plots for a certain switch size
since, as we saw above, the switch size slightly
affects the plots. We chose N of 1000. Figure 7 is anenlargement of Fig. 6(b), i.e., replots the curves withsmaller ranges of a, since it is difficult to tell the difference among the three lines from Fig. 6(b) when ais relatively small. Figures 7(a) and 7(b) illustrate theplots for 0 ≤ a ≤ 200 and for 200 ≤ a ≤ 1200, respectively. These three figures show that the proposed
network has equal or larger capacity than the folded
Clos network and that the proposed network has
equal or larger capacity than the unfolded Clos network if a ≤ 2.3N. In fact, the proposed network has up
to about 30% more capacity. For example, when thenumber of available switches a is 833, the capacity ofa folded Clos network is 125,000 with (k, l, m, n) =(500, 249, 2, 250), and that of an unfolded Clos network is 125,250 with (k, l, m, n) = (250, 333, 3, 501),
whereas that of the proposed network is 167,000 withNTT Technical Review

Vol. 19 No. 3 Mar. 2021

(k, l, m, n) = (500, 333, 2, 334). We also comparedcapacity using other switch sizes, N ∈ {200, 300, ...,
900} {2000, 3000, ..., 9000} and confirmed that the
proposed network has equal or larger capacity than an
unfolded Clos network if a ≤ 2.3N.
Finally, we compared the number of unused portson the left side of the input switch; those of a folded
Clos, unfolded Clos, proposed networks are N−2n,
N−n, N−n−ml, respectively. Figure 8 shows the number of unused ports when N = 1000. In the folded Closand unfolded Clos networks, almost half the ports
remained unused, whereas, in the proposed network,
the number of unused ports was less than 10 in morethan 90% of a ≤ 3000. The largest number of unused
ports was 134 when a = 1155. At this point (a =1155), m of the optimal connection parameters
decreased to 1 from 2, and this change made someports unused. We also compared the number ofunused ports using other switch sizes and obtained
similar results.
6. Related work
A Clos network is one of the most commonly used
non-blocking networks [1, 2, 6]. Many studies haveexamined non-blocking properties under various
switching environments. There are four non-blocking
properties; strictly [2], wide-sense [6], re-arrangeably[1], and re-packably [7]. The wide-sense property
allows us to select which path is used to establish a
new connection. Under the strict property, the path isarbitrarily chosen. The latter two enable us to rearrange existing paths to establish connections. We
focused on strictly non-blocking networks because
wide-sense non-blocking fails to significantly
60

600
500
400
300
200
100
0
50 100 150 200 250 300 350 400 450 500

Number of unused ports onleft side of input switch

Number of unused ports onleft side of input switch

Regular Articles

600
500
400
300
200
100
0
500

1000

1500

2000

2500

Number of switches

Number of switches(a) a  500(b) 500  a

3000

Proposed fP
Unfolded fU
Folded fC

Fig. 8. The number of unused ports on the left side of the input switch and number of switches used when N = 1000.

increase capacity if at all [6], and the latter two types
require the movement of existing connections, which
triggers network outages during path reconfiguration.
7. Conclusion
This article proposed a non-blocking network composed of square switches such as optical switches andconfirmed that it has a larger capacity than Clos networks. Our network has up to about 30% more capacity under reasonable parameters. Future work
includes investigating routing algorithms on our network and extensions to multiple intermediate layers.
References
[1]
[2]
[3]

V. Beneš, “Mathematical Theory of Connecting Networks andTelephone Traffic,” Academic Press, New York, 1965.
C. Clos, “A Study of Non-blocking Switching Networks,” Bell
System Technical Journal, Vol. 32, No. 2, pp. 406–424, Mar. 1953.
J. Turner and R. Melen, “Multirate Clos Networks,” IEEE Commun.
Mag., Vol. 41, No. 10, pp. 38–44, Oct. 2003.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

[4]

E. Oki, N. Yamanaka, K. Nakai, and N. Matsuura, “Multi-stage
Switching System Using Optical WDM Grouped Links Based on
Dynamic Bandwidth Sharing,” IEEE Commun. Mag., Vol. 41, No. 10,
pp. 56–63, Oct. 2003.
[5] H. J. Chao, Z. Jing, and S. Y. Liew, “Matching Algorithms for Threestage Bufferless Clos Network Switches,” IEEE Commun. Mag., Vol.
41, No. 10, pp. 46–54, Oct. 2003.
[6] F. Hwang, “The Mathematical Theory of Nonblocking Switching
Networks,” 2nd Edition, World Scientific, Vol. 15, 2004.
[7] W. Kabacinski, “Nonblocking Electronic and Photonic Switching
Fabrics,” Springer, 2005.
[8] A. Jajszczyk, “Nonblocking, Repackable, and Rearrangeable Clos
Networks: Fifty Years of the Theory Evolution,” IEEE Commun.
Mag., Vol. 41, No. 10, pp. 28–33, Oct. 2003.
[9] A. Jajszczyk, “A Dynamic Programming Approach to Optimization of
Switching Networks Composed of Digital Switching Matrices,” IEEE
Trans. Commun., Vol. 35, No. 12, pp. 1342–1346, Dec. 1987.
[10] A. Jajszczyk and W. Kabacinski, “Rearrangeable One-sided
Switching Networks Composed of Uniform Time-space Elements,”
Electron. Lett., Vol. 21, No. 7, pp. 285–286, Mar. 1985.
[11] R. Spanke, “Architectures for Large Nonblocking Optical Space
Switches,” IEEE J. Quantum Electron., Vol. 22, No. 6, pp. 964–967,
June 1986.
[12] A. S. Kewitsch, “Large Scale, All-fiber Optical Cross-connect
Switches for Automated Patch-panels,” J. Lightw. Technol., Vol. 27,
No. 15, pp. 3107– 3115, Aug. 2009.

61

Regular Articles

Toru Mano

Kimihiro Mizutani

Researcher, NTT Network Innovation Laboratories.
He received a B.E. and M.E. from the University of Tokyo in 2009 and 2011. He joined NTT
Network Innovation Laboratories in 2011. Hereceived a Ph.D. in computer science and information technology from Hokkaido University in2020. His research interests are network architectures and network optimization. He is a memberof the Operations Research Society of Japan(ORSJ).

Lecturer, Department of Informatics, Kindai
University.
He received an M.S. and Ph.D. in engineeringfrom the Nara Institute of Science and Technology, in 2010 and 2015. He was a researcher at
NTT Network Innovation Laboratories and NTT
WEST R&D Center from 2010 to 2019. His
research interests include future network architectures and various systems powered by deeplearning. He was a recipient of the Best Student
Paper Award from ICCSA in 2010 and the
Research Awards from IEICE in 2013. He is amember of the Institute of Electrical Engineers ofJapan (IEEJ).

Takeru Inoue

Osamu Akashi

Senior Researcher, NTT Network Innovation
Laboratories.
He received a B.E. and M.E. in engineeringscience and a Ph.D. in information science from
Kyoto University in 1998, 2000, and 2006. He
was an ERATO Researcher with the Japan Science and Technology Agency from 2011 to 2013.
His research interests cover algorithmic
approaches in computer networks. He was arecipient of the Best Paper Award of the AsiaPacific Conference on Communications in 2005.
He is a member of the Institute of Electrical andElectronics Engineers (IEEE).

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Research Professor, Research and Development Center for Academic Networks, National
Institute of Informatics.
He received a B.Sc. and M.Sc. in information
science and a Ph.D. in mathematical and computing sciences from the Tokyo Institute of Technology, in 1987, 1989, and 2001. He was a SeniorResearch Scientist in NTT Network Innovation
Laboratories from 2001 to 2018. His researchinterests are in distributed systems, network
management, and network architecture. He is amember of the Association for Computing
Machinery (ACM), Information Processing
Society of Japan (IPSJ), and Japan Society for
Software Science and Technology (JSSST).

62

Global Standardization Activities

Recent Standardization Activities in
ITU-T on Single-mode Optical Fiberand Space Division Multiplexing
Technologies
Taiji Sakamoto, Kazuhide Nakajima, and Noriyuki Araki
Abstract

Optical fiber standards have been discussed in the International Telecommunication Union - Telecommunication Standardization Sector (ITU-T). Various standards (Recommendations) have been developed based on discussions at Study Group 15 and are revised according to the progress in the opticalfiber telecommunication system. In this article, recent standardization activities in ITU-T on singlemode optical fiber (SMF) are described. Also, recent discussion toward standardizing space divisionmultiplexing technologies, which are promising for overcoming the capacity limit of SMF, is introduced.
Keywords: single-mode optical fiber, space division multiplexing, ITU-T SG15

1. Introduction
The International Telecommunication Union Telecommunication Standardization Sector (ITU-T)
has developed many standards (Recommendations)to specify telecommunication system requirements,
functions, and measurement methods to ensure theinteroperability and quality of network services.
Development of optical fiber standards is important,particularly for telecom operators, since transmission
lines can consist of optical fibers provided from multiple vendors. Therefore, standardization on transmission and geometrical characteristics of optical fibers
are mandatory to ensure the interoperability of thetelecommunication system. Table 1 shows the list of
ITU-T Recommendations related to optical fibers.
They are being developed under the responsibility of
Study Group 15 (SG15) [1] Working Party 2 (WP2)
Question 5. There are six types of Recommendations
regarding single-mode optical fiber (SMF) (G.652–
G.657). Question 5 has recently been discussing therevision of existing fiber Recommendations in accordance with the progress in optical fiber transmission
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

technologies. Recommendations G.650.1–650.3,
which are standard test methods for optical fibers, arerevised in line with the revision of the fiber Recommendations. The G series Supplements are the supplemental documents that provide beneficial information for Recommendation users. The International
Electrotechnical Commission (IEC) also has documents for optical fiber standards to describe fiberproduct specifications and has developed them inconjunction with the revision of Recommendations in
ITU-T.
2. Recent activities on SMF Recommendations
The Recommendations shown in red in Table 1 arethose being actively discussed. The G.652 fiber isused worldwide and recognized as “standard SMF.”
The G.657 fiber has optical characteristics compatible with those of G.652 fiber but has improved bending loss. These two fibers support transmission over
the O–L band* (1260–1625 nm) and used for various
applications such as access, metro, and core networks. Recommendation G.654 is for a fiber supporting63

Global Standardization Activities

Table 1. List of optical fiber Recommendation series.
Category

Fiber

Test method

Supplement

Document No.

Title

G.651.1

50/125 µm multimode graded index optical fibre cable

G.652

Single-mode optical fibre and cable

G.653

Dispersion-shifted single-mode optical fibre and cable

G.654

Cut-off shifted single-mode optical fiber and cable

G.655

Non-zero dispersion-shifted single-mode optical fiber and cable

G.656

Fibre and cable with non-zero dispersion for wideband optical transport

G.657

Bending-loss insensitive single-mode optical fibre and cable

G.650.1

Definitions and test methods for linear, deterministic attributes of single-mode fibre and cable

G.650.2

Definitions and test methods for statistical and nonlinear related attributes of single-mode fibre and cable

G.650.3

Test methods for installed single-mode optical fibre cable link

G.Sup.40

Optical fibre and cable Recommendations and standards guideline

G.Sup.47

General aspects of optical fibres and cables

G.Sup.59

Guidance on optical fibre and cable reliability

C–L-band* transmission and mainly used for submarine long-haul transmission systems. The revision ofthese Recommendations are active topics in ITU-T
due to the capacity growth in terrestrial and submarine optical fiber networks. In the next section, recent
activities for revising these SMF Recommendations
are introduced.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Value

Zero dispersion wavelength01300–1324 nm

Dispersion slope at 0
S0< 0.092 ps/nm2 × km(a) Previous specifications

24
Chromatic dispersion(ps/nm/km)

2.1 Revision of Recommendations G.652/G.657
In this section, recent revisions to Recommendations G.652 and G.657 are introduced. ITU-T fiber
Recommendation specifies, for example, fiber attenuation, mode field diameter (MFD), chromatic dispersion, or bending loss characteristics for satisfying
the transmission-system requirements and for ensuring interoperability. The demand for specifying moredetailed specifications has increased owing to theincrease in the bitrate of transmission systems. Particularly, the wavelength properties of each parameterare becoming important because wavelength divisionmultiplexing (WDM) transmission is an indispensable technology toward ultrahigh capacity transmission. From this background, ITU-T revised Recommendation G.652 in which the detailed wavelength
property of chromatic dispersion for G.652.D fiber is
specified. The revised specifications are shown in
Fig. 1. Figure 1(a) shows the previous chromatic dispersion specifications in which the range of the zero
dispersion wavelength λ0 range and maximum dispersion slope at λ0 are specified. The revised specifications for chromatic dispersion are now described in
the latest Recommendation, i.e., G.652, which specifies the maximum and minimum dispersion values

Parameter

20
16
12
8
4
0
−4
−8

1460nm1460
nm1300 1350 1400 1450 1500 1550 1600
Wavelength (nm)(b) Revised specifications

Fig. 1. C hromatic dispersion specifications for G.652.D
fiber.

over the wavelength range of the O–L band, as illustrated in Fig. 1(b). The maximum or minimum dispersion boundary for 1260–1460 or 1460–1625 nm is*

O-band (original band: 1260–1360 nm), C-band (conventional
band: 1530–1565 nm), and L-band (long-wavelength band:
1565–1625 nm).

64

Global Standardization Activities

SMF

G.652.D(0.30 dB/km)

Cut-off shifted fiber
for submarine network

G.654.D(0.20 dB/km)

Cut-off shifted fiberfor high-speed
terrestrial network

G.654.E(0.23 dB/km)
8

9

10

11

12

13

14

15

MFD (µm)

Fig. 2. Comparison of attenuation coefficient and MFD for G.652.D and G.654 fibers.

specified using an appropriate fitting function. This
revision provides the full O–L-band chromatic dispersion characteristics of G.652.D fiber and helps
system operators design the detail system configuration and requirements for future high-speed transmission systems. Since G.657.A fiber has compliant
transmission, except for having low bending loss, thesame chromatic dispersion specification was introduced in Recommendation G.657.
2.2 Revision of Recommendation G.654
Recommendation G.654 has been developed foruse mainly in submarine networks. The features ofG.654 fiber are low attenuation coefficient and large
MFD compared with those of G.652 fiber, and transmission over low-loss transmission window, i.e., theC–L band, to support long-distance transmission
systems. Figure 2 summarizes the specified fiber
attenuation coefficients and MFDs for G.654 subcategories. The loss attenuation and MFD for G.654
fiber are larger than those for G.652.D fiber. These
characteristics are effective for improving signal
quality (e.g. signal-to-noise ratio) by reducing thetotal span attenuation between repeaters or fiber
nonlinearity-induced noise. The G.654.D fiber has
the largest MFD and was developed for use in longdistance submarine networks. Although Recommendation G.654 was basically developed for submarineuse, low loss and large MFD are also being required
owing to the increase in the capacity and bitrate in
terrestrial networks. The G.654E sub-category wasestablished in 2016, which was for high-speed terrestrial core networks. The G.654.E fiber supports
more than 100-Gbit/s-capacity transmission with alower attenuation coefficient than that for G.652.D
fiber and narrower MFD specification range than thatfor the other G.654 fibers by taking into accountmulti-vender fiber connections and compatible bendNTT Technical Review

Vol. 19 No. 3 Mar. 2021

ing loss to ensure the applicability of the cable structure used in terrestrial networks.
2.3 Revision of Recommendation G.650.1
Recommendation G.650.1 specifies the test methods for optical fiber parameters such as attenuation
coefficient and cut-off wavelength. In 2020, Recommendation G.650.1 was revised owing to the recent
revisions of the fiber Recommendations in which theattenuation coefficient is specified by the maximum
value at a specific wavelength or wavelength range.
As fiber attenuation has wavelength dependency, it isimportant to know the loss spectrum to design a
WDM transmission system. To support this requirement, Recommendation G.650.1 describes a methodof estimating the attenuation spectrum from measured attenuation values at specific wavelengths.
Figure 3 shows a schematic diagram of this method,where a(λx) is the measured attenuation value at λx,
and attenuation values A(λ1)–A(λj) can be estimatedby multiplying the column vector [a(λx)–a(λz)] withthe estimation matrix. The number of rows or columns of the estimation matrix corresponds to thenumber of estimated attenuation values or measured
attenuation values, respectively. The figure also
shows an example when j attenuation coefficients are
estimated from the three measured attenuation values. It should be noted that the estimation matrix differs from the fiber product. Typically, 3–5 measured
values are required to estimate the O–L-band attenuation spectrum. For recently established Recommendation G.654.E, it is beneficial to know the attenuation spectrum for this type of fiber to design highcapacity terrestrial networks. The applicability of theestimation method described in Recommendation
G.650.1 for G.654.E fiber was recently investigated,
and the validity of the attenuation-spectrum estimation over the C–L band from two measured attenuation
65

Global Standardization Activities

A(1)

C12

C11

A(2)

C13

a(x)
a(y)

Cj1

A(j)

Estimated value

Cj 2

Cj3

Estimation matrixa(z)Measured value(a) Example of calculation equation for estimating j th value from three measure values

Loss

1–j

x
Measuredyz

Estimated(b) Schematic diagram for loss estimation

Fig. 3. Schematic image of attenuation-estimation method described in G.650.1.

values was confirmed, and Recommendation G.650.1
was revised accordingly.
In summary, the recent revisions of fiber Recommendations have focused on how optical characteristics can be specified for supporting terrestrial core orsubmarine long-haul networks with increased capacity or bitrate signals.
3. Recent activities on SDM standardization
Network capacity has been increasing at a rate of a
few tens of percent, and the capacity crunch with
SMF networks will become a serious issue in the
2020s. To overcome the capacity limit of SMF, fibers
for space division multiplexing (SDM) transmission
have been intensely investigated. Figure 4(a) shows
the conceptual images of SDM fibers. SDM fibers
can be basically categorized into two: multi-core
fiber or multi-mode fiber. Multi-core fiber has multiple cores within a cladding, and multi-mode fiber has
multiple propagation modes within a core. In SDM
transmission, multiple signals can be simultaneouslytransmitted through multiple cores or modes, achieving much higher capacity compared with that in SMF.
Before SDM fibers can be used in telecom networks
worldwide, it is necessary to establish an SDM fiber
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Recommendation in the same manner as the SMF
Recommendations. It was proposed and agreed at
ITU-T 2020’s January meeting to start discussion ona new technical report for SDM optical fiber andcable. Although the content of this technical report isunder discussion, it was agreed to include the related
topics on cable, splice/connectors, and installing
technologies. The main discussion pointes are: target
application and benefits of SDM technology and categorization of SDM fiber. Regarding the target application for SDM technologies, it is important to compare technologies that use SMF to improve spatial
density, such as high-fiber-count cable or reduced
coating-diameter fiber technologies, as shown in
Fig. 4(b). Although various SDM fibers have been
proposed, current multi-core fiber- or few-mode
fiber-based SDM fiber is being discussed as a potential candidate of SDM fiber. It is expected that thefiber parameters and test methods for such fibers will
be discussed and incorporated into this technicalreport. The tentative publishing year for this technicalreport is 2022. The discussion on SDM fiber standardization has been initiated in advance in Japan,
and the current technical level or challenges for SDM
standardization has/have been summarized as technical report-1077 entitled “Technical Report on Space66

Global Standardization Activities

200 µmCoating

Multiple cores within cladding

Multiple modes within a coreMulti-core fiber

High-fiber-count
cable

Cladding
Core

Reduced coating-diameter
SMF

Few-mode fiber(a) SDM fibers(b) SDM technologies with SMF

Fig. 4. SDM technologies discussed in ITU-T for developing new SDM technical report.

Division Multiplexing Technologies” (in Japanese)
published by the Telecommunication Technology
Committee (TTC) [2].
4. Summary
In ITU-T, the revision of SMF Recommendations
has focused on how to support the increased capacityand bitrate transmission system over 100 Gbit/s, andeach Recommendation has specified more detailed
optical characteristics of the fiber. Regarding theSDM fiber standardization, the technical report on
SDM fiber and cable, which will be published in
Taiji Sakamoto

Distinguished Researcher, NTT Access Network Service Systems Laboratories.
He received a B.E., M.E., and Ph.D. in electrical engineering from Osaka Prefecture University in 2004, 2006, and 2012. In 2006, he joined
NTT Access Network Service Systems Laboratories, where he has been engaged in research on
optical fiber nonlinear effects, low nonlinear
optical fiber, few-mode fiber, and multi-core
fiber for optical multiple-input multiple-output
transmission systems. Dr. Sakamoto is a memberof the Institute of Electrical and Electronics
Engineers (IEEE), The Optical Society (OSA),
and the Institute of Electronics, Information andCommunication Engineers (IEICE).

Kazuhide Nakajima

Senior Distinguished Researcher, NTT AccessNetwork Service Systems Laboratories.
He received an M.S. and Ph.D. in electricalengineering from Nihon University, Chiba, in
1994 and 2005. In 1994, he joined NTT AccessNetwork Service Systems Laboratories, where
he is engaged in research on optical fiber design
and related measurement techniques. He is also
acting as the rapporteur of Q5/SG15 of ITU-T.
Dr. Nakajima is a member of IEICE, IEEE, and
OSA.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

2022, is considered an important step in establishingan SDM fiber Recommendation. IEC has also decided to discuss the SDM fiber connector and amplifier
standardization. We believe that SDM standardization, including fiber, cable or any other related component technologies, will be actively discussed andproceeded in cooperation with ITU-T and IEC.
References[1][2]

ITU-T SG15, https://www.itu.int/en/ITU-T/studygroups/20172020/15/Pages/default.aspx
TTC TR-1077 (in Japanese), https://www.ttc.or.jp/document_db/
information/view_express_entity/1238

Noriyuki Araki

Senior Research Engineer, Access Media Project, NTT Access Network Service Systems Laboratories.
He received a B.E. and M.E. in electrical andelectronic engineering from Sophia University,
Tokyo, in 1993 and 1995. He joined NTT AccessNetwork Service Systems Laboratories in 1995,
where he researched and developed operationand maintenance systems for optical fiber cable
networks. He has been contributing to standardization efforts in ITU-T SG6 since 2006. He was
the rapporteur of Question 6 of ITU-T SG6 from2006 to 2008 and the rapporteur of Question 17
of ITU-T SG15 from 2008 to 2012. He also
served as the chairman of the ITU-T Focus
Group on Disaster Relief Systems and Network
Resilience and Recovery. He has been the vicechairman of ITU-T SG15 since 2013. He also
contributes to the activities of IEC TC86 (fiber
optic systems). He received the ITU-AJ award
from the ITU Association of Japan in 2012. He isa member of IEICE.

67

Short Reports

Simulating Complex Quantum
Networks with Time Crystals

1. Introduction
Researchers at the National Institute of Informatics(NII), NTT, Osaka University, the Japanese-French
Laboratory of Informatics (JFLI), and Tokyo University of Science have discovered that time crystals canbe used to simulate complex quantum networks.
Crystals are widely present in our daily life in theform of solid materials, the atoms of which selforganize in regular patterns. While these arrangements tend to be periodic in space, a newly discovered type of matter also shows such regularity in thedimension of time. These exotic systems are calledtime crystals and can only be observed under special
circumstances. Although recent developments inquantum technology and the excellent controllabilityof quantum devices now allow researchers to explore
time crystals in the laboratory, little is known about
their properties and applications. This article, firstpublished in Science Advances [1], focuses on theapplication of tools commonly used in network theory to represent these systems that allows for a deeper
understanding of their structures. With this, researchers were able to represent the melting of a time crystal
in terms of networks and found the emergence ofspecial type of structures: scale-free networks. These
complex networks have the same structural properties
as today’s Internet or airplane networks. Therefore,
having a physical system that allows for its efficientsimulation in the lab has enormous implications in
the technology arena.
2. BackgroundTypical crystals emerge in nature as the atoms thatform a solid arrange themselves in a periodic regularstructure. When this occurs, a continuous translation
symmetry of free space is broken, as the system is nolonger invariant under the arbitrary translations of its
atoms. The breaking of such a translation symmetry
can also be done in the temporal dimension, giving
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

raise to time crystals. A recent example of this phenomenon is presented as a twice the time discrete
time crystal (2T-DTC). An arrangement of interacting
two-level systems (such as spins) are under a periodic
external drive that repeats at every time T. Timetranslation symmetry breaking occurs when the
dynamical response of the 2T-DTC shows a different
periodicity than the external drive, as it needs 2T to
go back to its initial position.
3. Research results
This research shows that complex network structures, such as scale-free networks illustrated in Fig. 1,
emerge from slowly melting a 2T-DTC. The authorsof this article used a model of a DTC that can beexperimentally constructed and showed that its structure can be melted by adding an error to the drive(which plays the role of increasing temperature).
Marta Estarellas from NII stated that, “Aided by a
network visualization technique, for the first time weshed new light on the mechanisms by which the time
crystal melts.” Victor Bastidas from NTT added that,“It became clear that, with an increasing value oferror, the melting of the time crystal behaves like aphase transition, something similar to the way crystals, such as a block of ice, typically melt.”
4. Methods
Using Floquet theory, one can capture the periodic
nature of time crystals and obtain information of thesystem at the relevant discrete points of its quantum
dynamics. William Munro from NTT stated that, “We
represented this information in terms of networks
using a percolation rule and analyzed how its connectivity changes as the time crystal melts due to anerror in the drive.” The use of their network visualization technique made it possible to observe the temporal connection of the time crystal’s quantum states forthe first time, allowing them to extract information on
68

Short Reports

Fig. 1. Scale-free network emerging from the melting of a 2T-DTC.

the properties of the crystal by analyzing the properties of the crystal’s associated network.

thing that will surely aid in the future study andunderstanding of quantum many-body systems.

5. Outlook

6. Funding

Quantum computers exhibit enormous power even
if they are small. Time crystals also present thisadvantage as they can embed exponentially large networks using a small time-crystal device. Marta
Estarellas stated that, “Potential applications of ourresults include the use of time crystals as a quantum
simulator to analyze the structure of very large complex networks present in our daily lives, somethingthat in the classical world would otherwise be impossible as it would require a huge amount of computing
resources.” Kae Nemoto from NII added that, “Put it
this way, using this method and with just several
qubits one could simulate a complex network of thesize of the entire worldwide Internet.” Their introduced techniques proved networks to be a valuable
tool to represent quantum complex systems, some-This research has been made possible thanks to the
support of the Japanese Ministry of Education, Culture, Sports, Science and Technology Quantum Leap
Flagship Program (MEXT Q-LEAP) JPMX
S0118069605, the MEXT KAKENHI Grant-in-Aid
for Scientific Research on Innovative Areas Scienceof Hybrid Quantum Systems grant no.15H05870, andthe JSPS KAKENHI grant no. 19H00662. This project was also made possible through the support of a
grant from the John Templeton Foundation (JTF
60478)*.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

Reference[1]

M. P. Estarellas, T. Osada, V. M. Bastidas, B. Renoust, K. Sanaka, W.
J. Munro, and K. Nemoto, “Simulating Complex Quantum Networks
with Time Crystals,” Sci. Adv., Vol. 6, No. 42, eaay8892, Oct. 2020.

*

The opinions in this publication are those of the authors and do
not necessarily reflect the views of the John Templeton Foundation.

69

Short Reports

Researcher’s comment
Finding Real Applications for Current Quantum Devices
Marta P. Estarellas
Global Research Center for Quantum Information Science,
National Institute of Informatics
Quantum mechanics has had a huge impact on
many areas of science by providing a whole newmathematical framework that helped – and keeps
helping – us to enhance our knowledge of nature
in a very accurate way. This theory is so powerfulthat its applications have soon reached the information science arena, giving rise to what could be– if not yet – a new technological revolution. The
fast development of quantum technology has
brought the field of quantum computing into theera of noisy intermediate-scale quantum (NISQ)
devices. Proving quantum supremacy in such
quantum devices has become a race between
large academic institutions and companies. Theirmain focus is to attain the quantum advantage
from the universal computation point of view. It

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

is however unlikely that,with their current size anderror rates, NISQ devices
will be able to outperformconventional computers.
Under the QLEAP
umbrella, NII and NTT
Basic Research Laboratories are pursuing the quantum advantage proof froma different perspective: thinking of NISQ devices
as dedicated systems. In our collaboration, we
focus on the design of small-scale quantum
devices with hardware-specific applications that
surpass the current classical technology, without
the need of granting for universality. With thisaim, we have successfully proposed DTCs as
dedicated quantum computers to simulate andanalyze the structure of very large complex networks present in our daily lives, something that in
the classical world would otherwise be impossible as it would require a huge amount of computing resources.

70Short Reports

Researcher’s comment
Melting Time Crystals to Simulate Networks
on Small Quantum Computers
Victor M. Bastidas
Theoretical Quantum Physics Research Group,
Quantum Science and Technology Laboratory,
NTT Basic Research Laboratories
Crystals, such as salt and diamonds, are periodic arrays of atoms in space that are pretty common in our daily life. It has recently been demonstrated that time can be crystalized as well and thecorresponding state of matter is known as a discrete time crystal (DTC). In contrast to spatial
crystals, DTCs break discrete translational symmetry and time. They are rigid and robust against
small perturbation and remain stable for a longtime, as solid crystals do. Although DTCs are
extremely popular in research and for the generalpublic, practical applications of this exotic stateof matter are lacking. In October 2020, we proposed a scheme to use DTCs to simulate a largescale network. Our work is published in theAmerican Journal Science
Advances and will open a
new avenue of research in
the field of time crystals.
We showed the possibility
to simulate large networks
with an exponential number of nodes by using small
quantum devices.
This achievement was
obtained through joint research with M. P.
Estarellas and Prof. K. Nemoto from NII, T.
Osada and Prof. K. Sanaka from Tokyo University of Science, B. Renoust from Osaka University, and W. J. Munro from NTT Basic ResearchLaboratories under the QLEAP program. Our
work not only proposes an application of DTCs
but opens a new direction of research. We envision that the methods and techniques developed
from our work will inspire further research on
global properties of systems out of equilibrium
and their intimate relation with networks.

For inquiries:
Public Relations, NTT Science and Core Technology
Laboratory Grouphttps://www.ntt.co.jp/news2020/2010e/201016a.
html

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

71

Short Reports

World’s Fastest Directly Modulated
Laser Exceeding 100-GHz Bandwidth

1. Introduction
NTT in collaboration with Fumio Koyama, professor at the Laboratory for Future Interdisciplinary
Research of Science and Technology, Tokyo Instituteof Technology, has developed a membrane laser thatuses an indium phosphide (InP) compound semiconductor on a silicon carbide (SiC) substrate with highthermal conductivity. This laser, the world’s first
directly modulated laser with a 3-dB bandwidth
exceeding 100 GHz, can transmit at 256 Gbits (256billion bits) per second over a distance of 2 km.
Directly modulated lasers are now widely used indatacenter interconnections, but their modulation
speed is limited, which has been a problem for further
increasing transmission capacity. Our membranelaser will enable us to respond to the expected
increase in traffic with a low-cost and low-powerconsumption solution and contribute to the development of a high-capacity optical transmission infrastructure that supports NTT’s IOWN (Innovative
Optical and Wireless Network). This research wasreported in Nature Photonics on October 19, 2020
[1].
2. Research results
To increase the relaxation oscillation frequency,
NTT researchers have focused on the optical confinement factor of the active region and developed amembrane laser on a silicon (Si) substrate with athermal oxide (e.g., silicon dioxide (SiO2)) film.
Membrane lasers have a large optical confinement
factor in the active region and are compact, making it
possible to develop directly modulated lasers withlow power consumption. Since such devices are fabricated on a low-thermal-conductivity SiO2 layer, thetemperature increase in the active layer due to current
injection is large. Even if the current is increased, therelaxation oscillation frequency saturates at about 20
GHz due to saturation of the differential gain.
NTT Technical Review

Vol. 19 No. 3 Mar. 2021

To suppress the increase in the active-region temperature, we fabricated InP-based membrane lasers
on a SiC substrate, which has a thermal conductivity
approximately 500 times higher than that of SiO2.
Since SiC has a lower refractive index than InP, theoptical confinement factor is almost the same as thatof the device on SiO2. The membrane laser was fabricated by direct bonding with ultrathin (40 nm) SiO2
between the InP layer and SiC substrate. Assuming a
100-mW heat source, the temperature increase in theactive region of a membrane laser with an active layer
length of 50 μm was significantly reduced from 130.9
to 16.8°C when the SiO2 thickness was reduced from
2 μm to 40 nm. The current at which the relaxation
oscillation frequency reaches its maximum was 5.5
mA for a membrane laser fabricated on SiO2/Si substrate. In contrast, with the membrane laser fabricatedon SiC substrate, we were able to increase the current
to 30 mA and obtain the world’s highest relaxation
frequency of 42 GHz and a 3-dB bandwidth of 60
GHz.
In addition, using optical feedback from the end
facet of the output waveguide, we designed the device
in which photon-photon resonance occurs at around
95 GHz. As a result, we obtained a 3-dB bandwidthof 108 GHz and succeeded in generating a 256-Gbit/s(256 billion bits per second) pulse amplitude modulation (PAM)-4 signal and transmitting it over a distance of 2 km (Fig. 1).
3. Future developments
The future will see the development of transmitterscapable of handling the next generation of Ethernet
standards with transmission capacity exceeding 1
terabit with four or eight arrays. The simultaneousachievement of low power consumption is expected
to suppress increases in the power consumption of
datacenters and supercomputers due to the anticipated increase in data volume.

72

Short Reports

Electrical input
Ib: 27.2 mA
256-Gbit/s PAM-4, BTB
256-Gbit/s PAM-4, 2 km

10−1

2 ps/div.
Optical output

BER

212-Gbit/s PAM-4, BTB
212-Gbit/s PAM-4, 2 km
10−2

HD-FEC

10−3
KP4-FEC
10−4

2

4

6

8

Received optical power (dBm)
After 2-km transmission

Fig. 1. G eneration of 256-Gbit/s PAM-4 signal and bit error rate (BER) measurement results for back-to-back (BTB) and
2-km transmission. At the BERs below the dash-dotted line, data can be transferred without error using forward error
correction (FEC).

Reference[1]

S. Yamaoka, N.-P. Diamantopoulos, H. Nishi, R. Nakao, T. Fujii, K.
Takeda, T. Hiraki, T. Tsurugaya, S. Kanazawa, H. Tanobe, T. Kaki­tsuka, T. Tsuchizawa, F. Koyama, and S. Matsuo, “Directly Modulated Membrane Lasers with 108 GHz Bandwidth on a High-thermalconductivity Silicon Carbide Substrate,” Nat. Photon., Vol. 15, pp.
28–35, Oct. 2020.

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

For inquiries:
Public Relations, NTT Science and Core Technology
Laboratory Grouphttps://www.ntt.co.jp/news2020/2010e/201020a.
html

73

Short Reports

Demonstration of a Novel Method toGenerate Chaotic Signals Using a
MEMS Oscillator
1. Introduction
NTT and Tokyo Institute of Technology have demonstrated for the first time a simple and genericmethod of generating chaotic signals using a microelectromechanical oscillator. Using chaotic signals in
information technologies, such as machine learningand secure communications, has been extensively
studied. It is important to develop an efficient methodof generating chaotic signals using on-chip devices.
A microelectromechanical systems (MEMS) oscillator is a promising device for this purpose and has
advantages of high integration capability and precise
electrical control. With our method, we succeeded ingenerating chaotic signals using the libration of a
MEMS oscillator with one-order of magnitude lower
voltage than previous methods. Our method alsoenables the integration of a chaos generator with standard MEMS devices, such as ultrasmall microphones
and sensors. The method enables machine learning of
the output data from these devices directly on a common semiconductor chip.
The above results were obtained though the collaboration of NTT, where the device fabrication andmeasurement were conducted, with Tokyo Instituteof Technology, where the data analysis was conducted on the basis of the theoretical calculations. The
results were published in Physical Review Letters on
October 23, 2020 [1].
2. Background
Chaos is a commonly observed phenomenon invarious physical environments. Its temporal change
looks apparently random, but also exhibits regularand deterministic behavior governed by simple lawsof nature. There have been various studies on using
the complex behavior of chaos in information technologies, such as machine learning, secure communiNTT Technical Review

Vol. 19 No. 3 Mar. 2021

cations, and random-number generation.
MEMS is a type of fine structure device that functionalizes the physical motion of the composing elements. MEMS is used in many practical systems suchas high-performance sensors, high-frequency filters,
and digital mirror devices. Producing chaotic signals
using a MEMS oscillator was proposed some years
ago as a promising technology for processing theoutput data from MEMS sensors, but this technology
has not progressed due to the problem that chaos generation using standard MEMS technologies requiresa large area of electrodes as well as high applied voltage up to several tens of volts. We propose a method
for solving this problem by using libration of a
MEMS oscillator. This method generates chaotic signals simply by applying two different frequencies ofelectric signals. The efficiency is so high that we do
not need high voltage or large electrode structures;
therefore, highly integrated devices with low operation voltage can be fabricated.
3. Device structure and achievements
The MEMS device used in the demonstration consists of a suspended mechanical oscillator called adoubly clamped beam. The device was fabricated byprocessing a piezoelectric semiconductor, with whichan alternate voltage is applied to induce mechanicalvibration. We confirmed that the voltage including
two frequencies generates libration, and aperiodic
oscillation starts when the motion becomes large(Fig. 1). Theoretical analysis confirmed that theobserved signal has the features expected for chaos.
The voltage required to generate chaos was only a
few volts, which is one order of magnitude smaller
than with previous methods. We succeeded in generating chaotic signals with such small voltage using asimple beam structure.

74

Short Reports

40 (d)

35

020
Sine component (mV)

Vibration amplitude (mV)(a)  = 6.5 kHz50(b)  = 5.4 kHz503520(c)  = 3.3 kHz−40
40 (e)
0−40
40 (f)

50
0

30
20

0

2.5

5

−40

−40

0

40

Cosine component (mV)Time (ms)Fig. 1. Dependence of measured mechanical vibration on the applied frequency difference (∆).

4. Outlook
The frequency of mechanical vibration in this study
is several megahertz, which is not high enough forpractical applications. Therefore, we aim to generatechaotic signals at higher frequencies. After succeeding in fabricating such high-frequency devices, wewill focus on practical applications of these devices

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

such as for reservoir computation and secure communication.
Reference[1]

S. Houri, M. Asano, H. Yamaguchi, N. Yoshimura, Y. Koike, and L.
Minati, “Generic Rotating-frame-based Approach to Chaos Generation in Nonlinear Micro- and Nanoelectromechanical System Resonators,” Phys. Rev. Lett., Vol. 125, 174301, Oct. 2020.

75

Short Reports

Researcher’s comment
Toward Emergent Computing
Ludovico Minati
Specially Appointed Associate Professor,
Institute of Innovative Research, Tokyo Instituteof Technology
Nature does not work like a digital computer.
The incredible complexity of the biological,
social, and physical world is created by phenomena that follow simple rules. These rules are universal and nonlinear. They can produce many
phenomena, including chaotic movements. Chaotic movements are irregular and can be synchronized in complex ways. Therefore, it is possible
to generate complex patterns such as those foundin the brain. These patterns can one day help generate and analyze information.
Chaotic motion is usually generated using elec-

tronic circuits. Many types
and components are possible and I have studied
them. However, the characteristics of electronic components are not ideal, e.g.,using a battery for a longtime wastes energy andvibration resonance is not
so sharp. Optical andmechanical devices fabricated on a nanoscale are
far superior to transistors.
In collaboration with NTT Basic ResearchLaboratories, we were able to develop a device
that can generate chaotic mechanical vibrations
very easily. Due to its very simple structure, many
operations are possible. We will continue to collaborate to create a network of such devices.

Researcher’s comment
Towards Nanosystem-based MechanicalBrainsSamer Houri
Research Specialist,
NTT Basic Research Laboratories
The fields of oscillator networks and network
dynamics are currently witnessing astonishingprogress both in terms of the theoretical understanding of such complex systems as well as theirimplementation in a variety of information processing applications such as artificial intelligence. The high computational load of conventional neural networks has prompted alternative
designs of network-based information processing
systems; among them is reservoir computation.
In applications in which fast throughput andlow power constraints are dominant, e.g., edge
computing, the computationally heavy task of
training a neural-network-based system is not
acceptable; therefore, neural networks are
replaced with nonlinear networks, i.e., a reservoir, the connections of which remain unmodified
and the task of training is greatly simplified.
Our research aims to create efficient reservoircomputation platforms using micro- and nanoNTT Technical Review

Vol. 19 No. 3 Mar. 2021

electromechanical systems(M/NEMS) as the infrastructure. MEMS and
NEMS-based devices provide an excellent tool forthe implementation of thehighly nonlinear functions
required for efficient reservoir computation.
We recently demonstrated a path towards low-power generation of chaos
and identified the region where the edge of chaos
exists. The edge-of-chaos region is highly suitable at separating its inputs in a high-dimensional
and abstract phase-space. Since such inputs represent information to be processed, the system istherefore capable of separating inputs into categories, such as distinguishing images or identifying patterns.
We foresee exciting possibilities for the use of
M/NEMS platforms for such applications, especially because they can be easily fabricated inlarge numbers to produce very high-dimensional
reservoirs, and are able to react to external stimuli, thus combining both the functions of sensors
and information processors.

76Short Reports

For inquiries:
Public Relations, NTT Science and Core Technology
Laboratory Grouph t t p : / / w w w. b r l . n t t . c o . j p / E / 2 0 2 0 / 1 0 / l a t e s t _topics_202010241444.html

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

77

External Awards
DCASE 2020 Challenge Judges’ Award
Winners: Yuma Koizumi, NTT Media Intelligence Laboratories;
Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, Kunio Kashino,
NTT Communication Science Laboratories
Date: November 1, 2020
Organization: DCASE (Detection and Classification of Acoustic
Scenes and Events) Community
For “The NTT DCASE2020 Challenge Task 6 System: Automated
Audio Captioning with Keywords and Sentence Length Estimation.”
Published as: Y. Koizumi, D. Takeuchi, Y. Ohishi, N. Harada, and K.
Kashino, “The NTT DCASE2020 Challenge Task 6 System: Automated Audio Captioning with Keywords and Sentence Length Estimation,” DCASE2020 Challenge, Mar.–July 2020.
Certificate of Appreciation
Winner: Seishi Takamura, NTT Media Intelligence Laboratories

Date: December 31, 2020
Organization: The Institute of Electrical and Electronics Engineers(IEEE) Region 10
For his dedicated services and commitment as the 2019 & 2020
IEEE Region 10 Treasurer.
IEEE Fellow
Winner: Tomohiro Nakatani, NTT Communication Science Laboratories
Date: January 1, 2021
Organization: IEEE
For his contributions to far-field signal processing for speechenhancement and recognition.

Papers Published in Technical Journals andConference Proceedings
Distributed Server Allocation Model with Preventive Starttime Optimization against Single Failure
S. Masuda, F. He, A. Kawabata, and E. Oki
Proc. of the IEEE 21st International Conference on High-Performance Switching and Routing (HPSR 2020), May 2020.
This paper proposes a distributed server allocation model with thepreventive start-time optimization against a single server failure. The
proposed model preventively determines the assignment of servers tousers under each failure pattern to minimize the largest maximum
delay among all failure patterns. We formulate the proposed model as
an integer linear programming problem. We prove the NP (nondeterministic polynomial time)-completeness for the considered problem.
The numerical results reveal that the proposed model reduces thelargest maximum delay compared to one baseline; it avoids instability caused by the unnecessary disconnection, which frequently
occurs in the other baseline.

Participating-domain Segmentation Based Server Selection Scheme for Real-time Interactive Communication
A. Kawabata, B. C. Chatterjee, and E. Oki
IEICE Transactions on Communications, Vol. E103-B, No. 7, pp.
736–747, July 2020.
This paper proposes an efficient server selection scheme in succes-

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

sive participation scenario with participating-domain segmentation.
The scheme is utilized by distributed processing systems for realtime interactive communication to suppress the communication
latency of a wide-area network. In the proposed scheme, users participate for server selection one after another. The proposed scheme
determines a recommended server, and a new user selects the recommended server first. Before each user participates, the recommended
servers are determined assuming that users exist in the consideredregions. A recommended server is determined for each divided region
to minimize the latency. The new user selects the recommended available server, where the user is located. We formulate an integer linear
programming problem to determine the recommended servers.
Numerical results indicate that, at the cost additional computation,
the proposed scheme offers smaller latency than the conventional
scheme. We investigate different policies to divide the users’ participation for the recommended server finding process in the proposed
scheme.

Algorithms for Distributed Server Allocation Problem
T. Sawa, F. He, A. Kawabata, and E. Oki
IEICE Transactions on Communications, Vol. E103-B, No. 11, pp.
1341–1352, November 2020.
This paper proposes two algorithms, namely server-user matching78

Papers Published in Technical Journals and Conference Proceedings(SUM) algorithm and extended server-user matching (ESUM) algorithm, for the distributed server allocation problem. The server allocation problem is to determine the matching between servers andusers to minimize the maximum delay, which is the maximum time
to complete user synchronization. We analyze the computational timecomplexity. We prove that the SUM algorithm obtains the optimal
solutions in polynomial time for the special case that all server-serverdelay values are the same and constant. We provide the upper andlower bounds when the SUM algorithm is applied to the generalserver allocation problem. We show that the ESUM algorithm is a
fixed-parameter tractable algorithm that can attain the optimal solution for the server allocation problem parameterized by the numberof servers. Numerical results show that the computation time of
ESUM follows the analyzed complexity while the ESUM algorithm
outperforms the approach of integer linear programming solved byour examined solver.

Power of Uninitialized Qubits in Shallow Quantum Circuits
Y. Takahashi and S.Tani
Theoretical Computer Science, Vol. 851, pp. 129–153, January2021.
We study uninitialized qubits, whose initial state is arbitrary andunknown, in relation to the computational power of shallow quantum
circuits. To do this, we consider uniform families of shallow quantum
circuits with n input qubits, O(logn) initialized ancillary qubits, andnO(1) uninitialized ancillary qubits, where the input qubits only act ascontrol qubits. We show that such a circuit with depth O((logn)2) can
compute any symmetric Boolean function on n bits that is computable by a uniform family of polynomial-size classical circuits. Since
it is unlikely that this can be done with only O(logn) initialized ancillary qubits, our result provides evidence that the presence of uninitialized ancillary qubits increases the computational power of shallow quantum circuits with only O(logn) initialized ancillary qubits.
On the other hand, to understand the limitations of uninitialized
qubits, we focus on sub-logarithmic-depth quantum circuits andshow the impossibility of computing the parity function on n bits.

Graph-based Regional NMF for Distributed Computing
T. Koshizuka, K. Takeuchi, T. Matsubayashi, and H. Sawada
IPSJ Journal, Vol. 62, No. 1, pp. 387–396, January 2021.
Non-negative matrix factorization (NMF) is a popular unsupervised pattern recognition technique for the analysis of aggregateddata. In particular, non-negative multiple matrix factorization

NTT Technical Review

Vol. 19 No. 3 Mar. 2021

(NMMF) treats common elements from multiple data as common
factors, and execute simultaneous decomposition effectively. In thisstudy, we propose a novel matrix factorization method called regional non-negative matrix factorization (rNMF), which factorises multiple matrics simultaneously, focusing on physical relation between
aggregated data such as regional characteristics in addition to common factors. rNMF expresses data of physically close areas in asimilar feature space, and extracts intuitively interpretable bases andcoefficients from multiple matrics. The information of regional location is given by a graph. Furthermore, by solving the graph coloring
problem heuristically, rNMF works at high speed on a distributed
system even if the analyzed data are large matrices. In this paper, weformulate rNMF as an extended version of NMF and derive multiplicative update rules for parameter estimation. We performed experiment with real data, which were aggregated by region, in order toverify that rNMF can expresses adjacent regional data in a common
feature, rNMF attained similar generalization performance as theoriginal NMF, and rNMF works at high speed on a distributed system.

Optimal Server Selection Scheme with Optimistic Synchronization for Delay Sensitive Service
A. Kawabata, B. C. Chatterjee, and E. Oki
Proc. of the 18th IEEE Consumer Communications & Networking
Conference (CCNC 2021), Virtual conference, January 2021.
In distributed processing for communication services, a proper
server selection scheme is required to suppress delay by ensuring theevent occurrence order. Although a conservative synchronization
algorithm (CSA) has been used in this issue, an optimistic synchronization algorithm (OSA) can be a potential candidate for synchronizing distributed systems. In comparison with CSA, which reproduces events in occurrence order before processing application, OSA
can be feasible to realize low delay communication as the processing
events arrive sequentially. This paper proposes an optimal server
selection scheme considering OSA for distributed processing systems to minimize end-to-end delay under the condition that the holding time for application status is limited. In other words, the endtoend delay is minimized based on the allowed rollback time forapplication design or quality-of-service. Numerical results indicatethat the delay of the proposed scheme can be reduced by up to aquarter compared to that of the conventional scheme that is based on
CSA.

79