NTT Technical Review

September 2020 Vol. 18 No. 9

View from the TopTeruyuki Kishimoto, Executive Vice President, NTT WEST

Front-line ResearchersYutaka Miyamoto, NTT Fellow, NTT Network Innovation Laboratories

Feature Articles: Digital Twin Computing for Advanced Interaction between the Real Worldand Cyberspace

Digital Twin Computing Initiative

Challenges Facing Human Digital Twin Computing and Its Future Prospects



Digital Twin Computing of Things Opens Up a New SocietyApproaches to Cyber-physical Interactions Linking the Real World and CyberspaceSocial Issues with Digital Twin Computing

Regular Articles
Spatio-temporal

Equalization

Technology

forHigh-capacity

Underwater

Acoustic

Communication
Global Standardization Activities

Meeting Report on ITU-T FG-AI4EE—ICT Applications Using Artificial Intelligence in
Climate Change Countermeasures and Their Standardization Trends

External Awards/Papers Published in Technical Journals and Conference Proceedings

View from the Top

Live Under the Same Roof andPass Down Tacit Knowledge
for NTT WEST to Become aSocial ICT Pioneer
Teruyuki Kishimoto
Executive Vice President, NTT WEST
Overview

According to a survey conducted by the Tokyo Chamber of Commerceand Industry in early June during the novel coronavirus pandemic, 67.3%
of small- and medium-sized enterprises based in Tokyo have implemented
teleworking (remote working). Companies that have not yet introduced
teleworking have cited changing internal systems and ensuring security as
issues preventing implementing teleworking and have asked the government for support (relaxation of requirements for subsidy programs, expansion of target costs covered by such programs, etc.) and to provide usage
examples of teleworking. We interviewed Teruyuki Kishimoto, executive
vice president of NTT WEST, about how the company should respond tothese needs and the outlook and attitude of the company, which has reacheda turning point since its founding 20 years ago.
Keywords: ICT, remote-friendly, digital transformation

NTT WEST has achieved its highest profit since
inception 20 years ago—First, could you tell us about the current businessenvironment concerning NTT WEST?
NTT WEST marked its 20th anniversary in July
2019, and achieved its highest operating profit since
its founding, i.e., 132.2 billion yen, in fiscal year
2019 (April 1 2019 to March 31, 2020). I was delighted to welcome our 21st year with our employees who
have worked together with the aim of pulling ourcompany from the red at the time of our founding into
the black.
In 2019, the G20 was held in Osaka, and disasters
such as heavy rainfall and typhoons, mainly in
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Kyushu but also in other regions, stuck westernJapan, our operating area. Under these circumstances, operating revenue has been declining mainly forland-line telephone services; however, the range ofthis decline is decreasing, and the long-awaited
increase in revenue is close at hand. On top of that,
our Hikari fiber-optic services exceeded our annual
target for a net increase of 200,000 subscribers, i.e.,
increasing 220,000 subscribers, and, for the first time,
corporate sales and new business fields accounted formore than half the total revenue. We were also able toreduce costs by thoroughly improving efficiency
through digital transformation (DX). I believe that wehave been able to achieve our highest profit thus far
as a result of these company-wide efforts.
However, as we all know, the spread of the novel
1

View from the Top

coronavirus is causing society as a whole to stagnate,
and I think that many of our stakeholders are worrying about our financial results in the 2020 fiscal year.
We can see into the future when a natural disaster willsubside and the affected area will recover. However,the cause of anxiety differs regarding this pandemic
in that we cannot see into the future because no one
can predict, for example, how long the battle with thevirus will last.
—Anxiety is certainly widespread today. How will
NTT WEST handle this as a designated public institution under the Disaster Countermeasures Basic Act?
Under these difficult circumstances, we are striving
to take the lead in solving social issues using thepower of information and communication technology(ICT) to become a “social ICT pioneer.” We will continue transforming ourselves to become a trusted
company respected by the community and play a role
as a “vitamin” that makes the community vibrant.
We are addressing people’s anxiety by introducing
three key phases. The first phase is “business continuity and safety.” As a designated public institution, we
have a mission to fulfill requests for improving thecommunication environment for accessing local government, companies, medical care, and public services even under today’s difficult circumstances. It isalso very important to balance that mission with due
consideration for the health and anxiety of ouremployees. Regarding this balance, we put a system
in place, which was prepared as a countermeasureagainst a new strain of influenza (H1N1) that spread
about ten years ago. The present pandemic is the firsttime we have called upon this system, and we areworking to maintain business continuity and thehealth and safety of our employees through measures
such as introducing company-wide teleworking andstaggered working hours.
The next phase is “recovery from confusion.” In
response to the resumption of social activities, wewill endeavor to solve a variety of social problems
based on the keywords “remote” and “online” bymarshaling the products of the NTT WEST Groupand assuming requests from customers to expand
coverage. Moreover, I believe that these efforts will
not only extend across the NTT WEST Group but will
harness the power of the NTT Group, which will leadto optimizing the entire Group.
When I joined the company, I was at the forefrontof my field. Frontline workers are needed when adisaster strikes or problems occur, and I feel that our
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

frontline employees are NTT’s “face” for directly
communicating with customers. The expectations
that customers have toward NTT are high. With theseexperiences in mind, I think our important tasks are
how we can do our best in the event of a disaster orother difficult situations and how we can fulfill our
mission as a designated public institution. All departments and employees must also listen to customers
and respond to their requests in the same manner as
frontline employees.
The last phase is “growth.” The reform of work
styles and creation of a remote-friendly society—with a focus on the new normals of “during corona”
and “after corona”—are ongoing. As remote working
becomes more widespread, Japan’s businesses,
which are currently over-concentrated in Tokyo, will
be dispersed. In this transitional era, the ICT infrastructure is considered to be the blood vessels ofsociety. As well as providing traditional network services and solutions, through cybersecurity and otheruser support, we will strive to ensure that customers
will be able to use our services safely and in a stressfree manner.
To give an example of our efforts, ELGANA, abusiness service that provides a secure business communication environment, enables an efficient way ofworking regardless of time and place. As of the end ofJune 2020, the number of subscriber identifiers registered for ELGANA has already exceeded 50,000. We
are planning to enhance services that can meet a wide2

View from the Top

range of customer demands concerning, for example,
safety and security, for promoting a remote-friendly
society. The spread of the novel coronavirus is a crisis; however, we can think of it as taking up a newchallenge.
How to attract loyal customers in a rapidly
advancing society—How is the new medium-term business plan being
affected by the novel-coronavirus pandemic?
We are conducting business while taking steps toprevent the further spread of the novel coronavirus. In
our New Year’s in-house presentation, the trend in2020 was compared to an ice-hockey stick. At the
beginning of 2020, we considered ourselves at a pointof an upturn, namely, the contact point between theshaft (handle) and the blade (the part that hits thepuck) of the stick. Therefore, we started 2020 statingthat we would like to follow that upward trend.
Despite the unprecedented situation brought by the
pandemic, I am convinced that we are steadily progressing to increase revenue.
Under our new medium-term business plan, we aim
to achieve operating revenue of 1.5 trillion yen by2025, an operating income ratio of 10%, and 10 million subscribers of Hikari fiber-optic services. The
year 2020 is very important. First, to become a social

ICT pioneer, we aim to be a trusted company that
continues to be the choice of local customers. We will
endeavor to contribute to making local communities“smart” (smartization) by focusing on customer success, that is, helping customers to achieve their business goals by carefully responding to their requests
and challenges.
Ever more speed is now demanded, and progress ismeasured in seconds and minutes, rather than days.
Therefore, the period from when we reform our work
styles to improve efficiency to providing the results ofthat reform as a product use case to customers is
shortening. We believe that if we can meet thisdemand swiftly, increased sales figures will follow. I
think it is natural to provide solutions specialized for
ICT and important to know how to add value to themand use conventional infrastructure. Moreover, thefailure of communication equipment should theoretically occur once every ten years. However, natural
disasters have been occurring one after another, andcommunications have been disrupted in the affectedareas. If we don’t take these types of problems affecting our customers seriously, customers will immediately become anti-NTT. However, if we work sincerely and improve, they will become loyal customers. Repairing equipment and fixing faults is one of
our important successes, i.e., satisfying our customers. I’d like us to keep responding sincerely and gaining loyal customers regardless of the circumstances.
The important duty of decision and execution—Could you tell us what is important to you as asenior manager in such an unprecedented situation?
I think all our employees and customers are worried. Although the sources of those worries are varied, I think it is important to try to dispel them. Whilethe current situation is eliciting an emergency
response, I think we should show them that we aremoving toward the future beyond this emergency.
The department concerning facility planning addresses matters from a long-term perspective and creatinga grand design of telecommunication infrastructure.
For example, the Innovative Optical and Wireless
Network (IOWN) is aimed for commercial application in 2030, and the Osaka Expo, which will be heldin 2025, will be a test case towards the commercial
application of IOWN. The facility planning department is also required to improve efficiency. It will benecessary to streamline and simplify the large amountof equipment and facilities and switch to new services.

NTT Technical Review

Vol. 18 No. 9 Sept. 20203

View from the Top

I believe that senior managers are required to create
an environment in which employees can talk about
their long-term goals while keeping practical necessities in mind.
To create such environments, we, senior managers,
have two important duties, namely, decision andexecution. You may not achieve 100% (i.e., perfection) in one go, or you may make a detour instead oftaking the shortest distance. Even so, I feel that weneed to make decisions and steadily implement them.
I think that not only myself but all senior managers
are solving problems at our respective managerial
positions every day. For example, even if the medium-term business plan that we are currently working
on takes us in the right direction, we may sometimes
not be able to make judgements and decisions. I think
it is my job to make those decisions and encourageothers at such times. What’s more, as I mentioned
earlier, progress is accelerating, so rather than waiting to find the means to achieve 100%, I’d like to startwith 50 or 60% and make fine adjustments throughcollective effort.
—Senior managers taking responsibility for decisions is assuring. How are bonds between you anddepartment managers created?
To address the situation concerning the novel coronavirus, I have a conference call every day and a faceto-face meeting at least once a week. It is importantfor department managers to understand what is happening and share that knowledge. In fact, this morning’s meeting followed what my superior was doingaround 2011. I have also drawn on my experiences inprojects such as the Kyushu-Okinawa Summit in
2000, and I’m glad that my colleagues who sharedthat past with me are leading the present, and mypresent colleagues have inherited that knowledge. To
that end, I’d like to create an environment in which
employees inspire each other. There is a saying, “liveunder the same roof,” which expresses exactly what
we are doing. Previous employees built telephoneexchanges and pulled cables during the period ofJapan’s rapid growth (from 1954 to 1970). Thesefacilities are about to be renewed. It is natural that theway of thinking back then and the way of thinking
now differ. However, the process of technology
migration requires understanding the thoughts andbackground concerning those days and drawing a
grand design for the future. Goods, construction
methods, and DX and other tools have all changed;
however, the important things have not.
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

About 10 years ago, we established the Gold
Meister system at NTT WEST to maintain a traditionof craftsmanship. That is because our senior employees have tacit knowledge and skills that match customers’ businesses and the situation at work sites.
However, they neither express such knowledge andskills in spoken word nor write them down. Therefore,
we get our young employees to ask their seniors
many questions so that tacit knowledge can be passed
from person to person. Young employees absorb
information like sponges, so they are better at
responding to ever-changing technologies and services; therefore, I think that we are making good useof one another’s characteristics.
Always keep your antenna up to spotsocial trends—Please tell us what you want from researchers andengineers inside and outside the company.
You need to go out and see the world so that theJapanese/Chinese saying, “The frog in the well
knows nothing of the great ocean (i.e., He that stays
in the valley shall never get over the hill)” will not
ring true. Technology has changed drastically since
the land-line era. Even elementary-school students
are making apps. In the era of open technologicaldevelopment, it is necessary to keep your antennaraised and know how companies and people otherthan telecommunications carriers are doing business.
NTT Group’s basic research is our backbone. I hearfrom many customers that this research is “fantastic.”
I feel that we can provide customers with products
and services with confidence thanks to the efforts ofour research laboratories. In 2020, NTT WEST established a technology strategy with the following fivepillars: (i) further promotion of local business, (ii)4View from the Top

promotion of DX/data-utilization business, (iii) promotion of carrier’s carrier business, (iv) initiatives for
the IOWN era, and (v) initiatives for the migrationfrom the public switched telephone network to theInternet protocol network.
We are an operating company, so our role is not justselling our products but also ensuring our products beused properly and will not easily fail. Even if our
products fail, it is important to keep the repair time as
short as possible through remote support. I’d like you
to pursue technological development that supports
our core operations and maintenance work whileadvancing the research that supports troubleshooting.
I’d also like researchers of the NTT Group to continue their efforts to overcome the global competition in
research and development (R&D). I’d also like to see
these efforts organically connected and develop intoa business.
Since research and telecommunication are advancing rapidly, I want to make a road map and work on
what to do next and with whom to form alliances by

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

interacting with the outside world and understandingsocial trends. I believe that setting goals and runningtowards them will certainly lead to further trust in the
NTT Group and organically connecting our businesswith R&D.

Interviewee profile
Career highlights
Teruyuki Kishimoto joined NTT in 1986 andbecame president and CEO of NTT Field Technoin 2014. In 2017, he became a member of theboard of NTT WEST, senior executive manager
of Kansai Regional Headquarters, and generalmanager of the Osaka Branch. He has been
executive vice president and concurrently executive manager of Network Department, Plant
Headquarters since June 2019.

5

Front-line Researchers

Sharing Moments of “This Is It!”
with Colleagues to Research andDevelop Optical Communication
Technology for Revolutionizing
Society
Yutaka Miyamoto
NTT Fellow, NTT Network Innovation
Laboratories
Overview
The development of information and communication technology (ICT) has revolutionized people’s social lives. Amidst thecurrent unprecedented situation due to the worldwide spread of
the novel coronavirus, services and applications that use ICT, suchas teleworking and online diagnosis, have been supporting people’s lives and economic activities. According to the White Paper
on Information and Communications in Japan issued by the Ministry of Internal Affairs and Communications, the number ofInternet-of-Things devices installed worldwide is said to reach 40
billion in a few years. We asked NTT Fellow Yutaka Miyamoto,
NTT Network Innovation Laboratories, about his research and development on the optical communication infrastructure that supports these ICT services and application and path to its practical applicationand his attitude as a researcher.
Keywords: high-capacity optical communication, capacity crunch, scalable optical-communication
technology

Challenge to overcome the capacity crunch—Please tell us about the research you are currently
undertaking.
NTT has been a world leader in the research anddevelopment (R&D) of optical communication technology. We started practical application of time-division multiplexing (TDM) optical-fiber transmission
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

systems in 1981. By triggering three paradigm shifts
concerning optical transport systems, namely, thedevelopment of optically amplified transmission systems, wavelength-division-multiplexing (WDM)
systems, and digital coherent systems, we haveincreased transmission capacity by about 106 times
over the past 40 years. The amount of data traffic
continues to increase at an annual growth rate of6

Front-line Researchers

Scalable optical-communication technology(bit/s)

10P
1PTransmission capacity per fiber (> 50 km)

AllPhotonics

D

R&

100T

m

m

Co

y

ls

cia

er

m NetworkstePhysical limit of currently
used optical fiber (SMF)(capacity crunch)

10T(3) Digital coherent

1T
100G

(2) WDM

10G(1) Optical amplification (EDFA)

1G

Electrical multiplexing (TDM)
Electrical communication
Optical communication: Cutting-edge research

100M
10M

SMF

: NTT commercial system

1980

1990

2000

2010
Year

EDFA: erbium doped fiber amplifier

2020

2030
Coaxial cable

Fig. 1. Challenge to overcome the capacity crunch by using scalable optical-communication technology.

about 1.4 times, and as the full-scale introduction of
fifth-generation mobile communication systems (5G)
and the Internet of Things progresses, it is expected tocontinue to increase exponentially [1].
A high-capacity WDM system using digital coherent detection with a channel capacity of 100 to 400
gigabits per second (Gbit/s) per wavelength has
recently been developed and installed in our network,
enabling high-capacity long-distance transmission
with the total system capacity of 8 terabits per second(Tbit/s) via a single fiber [2, 3]. Through further
upgrading of the digital coherent system, a channel
capacity of 600-Gbit/s per wavelength will soon becommercially available. It is expected that highspeed transmission exceeding 1 Tbit/s per wavelength
will be achieved by pursuing cutting-edge R&D [3,4].
If the development of the optical-communication
infrastructure over the next 10 years is predictedbased on the trend concerning commercial systems
shown in Fig. 1, it becomes clear that long-distancetransmission with petabit-per-second (Pbit/s)-classcapacity will be required by the 2030s. According torecent research, however, the ultimate physical limitof transmission capacity in long-distance transmission using an installed optical fiber, that is, singlemode fiber (SMF), has become apparent near 100
Tbit/s—the so-called capacity crunch. The R&D thatNTT Technical Review

Vol. 18 No. 9 Sept. 2020

we are conducting to overcome this capacity crunchand create a petabit-class optical communication
infrastructure that can economically accommodate
100 times or more the current data traffic while ensuring low power consumption is called scalable optical-communication technology [1]. To achieve this, I
believe that we must pursue technological innovationthat combines the optical transmission technologythat we have been developing with new transmission
medium technology (optical fiber). In other words, afourth paradigm shift is necessary.
As an example of such R&D, working closely withthe optical materials and devices research group at
NTT, we are researching a space-division multiplexing (SDM) optical-communication technology that
can increase transmission capacity per optical fiber
exceeding 1 Pbit/s, which is more than 125 times thatof a practical system using SMF. In 2012, in collaboration with domestic and overseas research institutes,
we experimentally demonstrated a transmission
capacity of 84 Tbit/s per core, namely, transmitting
optical signals at 1.01 Pbit/s per fiber (i.e., 84 Tbit/s
× 12 cores) over 52.4 km. This was the world’s first
1-Pbit/s transmission experiment by using a multicore fiber (MCF) housing with 12 cores (i.e., paths of
optical signals transmitted within the optical fiber) in
one optical fiber and by wavelength multiplexing 32
quadrature-amplitude modulation (32QAM) signals
7

Front-line Researchers

Channel capacity B = Spectral efficiency  × Signal bandwidth Bs × Number of channels in SDM/WDM M
Signal bandwidth Bs
150 GHz

10T(target)(a) Ultra-wideband opto-electric conversion 80 GHz
of signals using a hybrid integrated optical
front-end module

1.3T(our work as of 2019)

Bandwidth lim
itation of CMO
S integrated
circuits

50 GHzCurrent optical fiber (SMF)
800G

30 GHzMCF
15 GHzMulti-mode fiber &inlsMIMO-DSP for mode
ne
n
ha M
multiplexing
fc M 3r o /WDemb M 6
Nu SD
10

600G

100G
200G
1

QPSK 16QAM

64QAM(b) Reduction in DSP complexity
by optical parametric processing
based on PPLN

256QAM

Spectral efficiency 

DSP: digital signal processing
MIMO: multiple-input and multiple-output
QPSK: quadrature phase-shift keying

Fig. 2. Efforts to achieve 10-Tbit/s capacity per channel.

(an advanced higher-order QAM digital-coherent
technology). In 2017, we confirmed the feasibility ofa long-distance optically amplified transmission system over 1000 km at 1 Pbit/s using a 32-core MCF(Fig. 1).
To increase transmission capacity per wavelength(channel capacity), we are working closely with thedevice research group at NTT toward optical-transmission technology with 10-Tbit/s-class channel
capacity. For example, by using SMF, we demonstrated the world’s first long-distance WDM transmission system with a channel capacity of 1 Tbit/s,
which has been difficult to achieve with conventional
silicon complementary metal oxide semiconductor(CMOS) circuits and device-packaging technology.
In 2019, on the basis of a new opto-electronic integrated configuration of an ultrahigh-speed opticaltransceiver front-end circuit, we successfully demonstrated a long-distance WDM transmission systemwith the world’s fastest channel capacity of 1.3 Tbit/s.
This was enabled by integrating an analog-multiplexer integrated circuit (AMUX IC) (with bandwidth
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

over 100 GHz) and a broadband indium phosphide(InP) semiconductor modulator into an optical frontend module (Fig. 2(a)).
We are also researching and developing opticalparametric amplification technologies based on periodically poled lithium niobate (PPLN) that NTT
laboratories have been developing over several years.
These technologies dramatically reduce the complexity of digital signal processing and significantly
enhance the bandwidth of optical amplification andwavelength conversion, which are difficult to accomplish with conventional technologies only (Fig. 2(b)).
—You have been producing world-first achievements.
Do you feel that you are driving trends and are on the
cutting edge?
I feel that we have somehow caught up with thetimes. For the optical communication infrastructure
we have been researching and developing at NTT(Fig. 1), it took about 10 years from the time target
performance was first experimentally verified (yellow
8Front-line Researchersline) in the R&D stage until systems were installed incommercial networks (green line). The reason that somuch time is needed is that once the optical communication infrastructure is introduced, it must be maintained and operated over a system life of 10 years or
more. In other words, it cannot be removed immediately once installed, so careful consideration isrequired. During global competition in the advancedresearch phase when the target performance wasexperimentally demonstrated for the first time waswhen we developed a method for achieving thedesired performance. Since communication systems
cannot be created with one elemental technology,
researchers devise various elemental technologies
during the first half of the decade and repeatedly test
the system performance of various combinations ofsuch technologies. In the later practical-application
phase, through various efforts such as (i) determining
which combination of elemental technologies canwithstand the economic and reliability requirementsand (ii) investigating the possibility of internationallystandardizing the necessary components of the system while finding a development partner, we willcontinue to select our technology candidates.
Through these considerations, we have put the selected technologies into practical use. Technologies thatsurvive the original idea to be put to practical use
account for about 20 to 30% of the technologies weare researching; the remainder have been shelved. To
reach the final goal of practical application, it istherefore always important to have multiple technology portfolios regarding system development.
Various optical communication technologies developed for telecoms have recently started to be introduced into datacenters of corporations such as GAFA
(Google, Apple, Facebook, and Amazon), whodemand performance from a different perspective
than telecom applications. Since the life cycle oftechnologies for these corporations is short and theapproach to maintenance and operation differs significantly, the performance required by R&D and the
speed of R&D are also changing significantly. In
other words, R&D that matches the changing times isneeded.
Interestingly, I feel that the optical communication
infrastructure and new information services aredeveloping in a balanced relationship, like a chickenand-egg situation. I joined NTT in 1988, when thetelecommunication infrastructure mainly carried
telephone traffic. At that time, the paradigm shift in
optical communication systems using optical amplification technology began. I was fortunate to be able
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

to engage in R&D and practical application of a longdistance optically amplified transmission systemwith a total capacity of 10 Gbit/s using one wavelength per optical fiber. When such transmission was
first installed in a practical network in 1996, somepeople said that its capacity was too large for telephone traffic and it might be unnecessary to expandthat capacity further. However, in the late 1990s, as
the spread of the commercial Internet in Japan accelerated, Internet providers were appearing one after
another, and we entered an era in which transmission
capacity was insufficient, even in regional networks.
Under those circumstances, 10-Gbit/s optical-communication infrastructure was quickly introduced
throughout Japan and spread globally. I have hadmany similar experiences. The era of using smartphones as a matter of course has recently arrived, andto adequately accommodate the demands concerning
data traffic on the mobile Internet, the 100-Gbit/s per
wavelength optical communication infrastructureusing the digital coherent technology I mentioned
earlier was deployed in a timely manner. With the
spread of 5G services, which started commercial services in 2020 in Japan, it is assumed that the transmission capacity will be insufficient again as new services and industries are created. Therefore, I believe
it is necessary to continue our R&D to provide thecommunication infrastructure required by society ofa particular time.
Thus far, I couldn’t have predicted all the technological trends that I mentioned above. Rather, byperpetuating the R&D cycle and practical applicationof transmission systems that satisfy the current
required performance, we were able to catch up withcurrent performance demands. I think that the reward
of continuing this R&D cycle and practical application is being able to feel the huge impact that thesepractical optical fiber communication technologies
have on society. For example, due to the novel coronavirus pandemic, this interview is being conducted
via a web conference. Such real-time video communication, which many people now take for granted,
was unthinkable a short while ago. The importance ofthe telecommunications infrastructure is emphasizedin the event of such a sudden pandemic or natural
disaster, but this infrastructure cannot evolve overnight. Therefore, it is necessary to work diligently inresponding to development and future social infrastructure needs. I believe that only through steadyR&D can we provide technology that meets people’s
expectations in a timely manner.

9Front-line Researchers

Timely R&D, practical application, andconverting irrational ideas into common sense
are required—You have focused on creating robust technologythat can quickly meet the needs of the society of aparticular time. How did you actually feel on becoming an NTT Fellow? And please tell us what you havevalued in your research.
Playing my part to prevent the spread of the novel
coronavirus, I’ve been working from home, so I
haven’t really felt much about being an NTT Fellow
yet. I think that what NTT laboratories have been
constantly pursuing as a world leader on the cuttingedge from before I joined until now can be condensed
in the sentence, “Let’s conduct research by drawing
from the fountain of knowledge and providing specific benefits to society through practical applications” by Goro Yoshida, the first director of the Electrical Communication Laboratory of Nippon Telegram and Telephone Public Corporation (currentlyNTT).
In other words, I think this mantra has been passed
from generation to generation by mobilizing all kindsof “intellectual fountains” of NTT laboratories while
continuing the R&D cycle and practical applicationthat I have talked about. I think that one of my roles
as an NTT Fellow is to receive the baton that has
passed the fountain of knowledge from which ideas
and knowledge spring up like water from a fountainto the next generation through everyday R&D.
In the R&D cycle and practical application, common problems are identified in the first phase of R&D
and many irrational ideas are brought to light. Then,the next practical phase is the act of selecting the ideawith outstanding practicality and converting what
was born from an irrational idea into common sense.
By continuously repeating this process, we accumulate knowledge and ideas, creating the next ideas andinsights, that is, it becomes a fountain of knowledge.
I think it is important to develop and accumulate
ideas and knowledge in this manner so that they do
not wither. The words of Goro Yoshida I mentioned
above are a kind of motto to me as a researcher, andas an NTT Fellow, I think they will be a guide for me
to lead the people around me and implement ideas.
I remember that when I joined NTT, my superiors
taught me that the Japanese Kanji character “chi” has
two meanings. One is knowledge and the other is
wisdom. Although I didn’t really understand it at thattime, as I gained experience, I finally came to underNTT Technical Review

Vol. 18 No. 9 Sept. 2020

stand that knowledge is an insight into various fields
and wisdom is how to use that insight. As new people
enter your team, new knowledge will flow in. I think
making that knowledge into something usefuldepends on wisdom to apply that knowledge in atimely manner in accordance with the demands of thetimes. When we are engaged in R&D and practicalapplication while confronting the times and thetrends in technology, there are moments when we feelthe inspiration of wisdom and shout, “This is it!”
Although it is very difficult to express this feeling inwords, I think it is very important to not let such
moments get away and to focus on them. In my case,
I’ve experienced the process from R&D to practicalapplication about three times, and within those experiences, I’ve had moments in which I shared the feeling of “This is it!” with my research colleagues. This
moment is exactly like when the fountain of knowledge is gushing. That is, we can share our goals andvalues only by repeating trial and error through dailyexperiments and commercialization efforts together
with colleagues involved in the same R&D, and we
can feel that wisdom resonate between us through thephrases, “This is it!” and “That’s right!” I want toproceed with R&D by sharing such moments with as
many colleagues and development partners as possible.
About five years ago, I had the opportunity to collaborate with an overseas R&D partner. I experienced
that how Japanese people communicate with eachother did not work in such a collaborative situation.
At that time, I wondered how to handle such a situation and realized that no matter what country thepartner is from, the key to success is to share the same
goals and express them clearly in words. Even if there
are conflicting opinions, if you can share your goals
and values, you can return to the basics and overcome
those conflicts.
In the early 2000s, by applying communication
methods (e.g., error-correction codes and differential
phase modulation) that use the properties of lightwaves, we experimentally demonstrated the world’s
first long-distance WDM transmission with a capacity of 43 Gbit/s per wavelength. However, while conducting field investigations at the practical-development stage, we discovered that sufficient performance
could not be secured with a combination of elemental
technologies we prepared at the beginning. Under
this circumstance and to meet the performancerequired by a practical system, we were forced todevelop additional elemental technologies.
At that time, I was the project leader for the10

Front-line Researchersdevelopment of signal-processing large-scale integrated circuits (LSIs) for optical transport networks,
but I was pretty much overwhelmed by this situation.
Even so, we managed to launch elemental technology
prepared as future technology ahead of schedule andput it to practical use. In the real world, it is impossible to predict what will happen, and many thingscannot be understood without actually measuring
them in a real environment. It is often the case thatcertain tasks cannot be done by one person alone;
however, I had the experience that I could somehow
reach my goal even at such times if the technology
portfolio was prepared. I think that having the courage to take the next step in difficult situations andovercoming various difficulties faced thus far hasstemmed from being able to make the right decisions
by sharing objectives and wisdom with many outstanding colleagues and development partners andresonating with them through the phrases, “This isit!” and “That’s right!”.
To make a calm decision even in such a difficult
situation, it is sometimes important to rest the mind
and body to keep your feeling fresh. As I mentioned
above, I often feel run down when I don’t get thedesired results, so I may not be able to proceed without resetting my mind. For that reason, I think it’s
good to have your own reset mechanism, such as hobbies and meditation. I enjoy appreciating art. Above
all, I like Rene Magritte’s “The Large Family.” In
addition, Samuel Ullman’s timeless poetic masterpiece “Youth” gives me courage. A friend read me thefollowing line from that poem, “In the center of yourheart and my heart there is a wireless station; so longas it receives messages of beauty, hope, cheer, courage and power from men and from the Infinite, solong are you young.” As a researcher of telecommunications technology, I’m moved by the fact that theauthor uses an expression associated with wirelesscommunication, namely, a “wireless station” of theheart. When that poem was written about 100 years
ago, it was just before the first practical use of wireless communication enabling transoceanic communication, and it was the first time a maritime disaster—
the sinking of the Titanic—was wirelessly detected in
the middle of the Atlantic Ocean, so lives were probably saved by this first use of wireless technology.
Knowing the importance of wireless communication,
which enabled global real-time communication for
the first time, was recognized around the world andused as a metaphor for works of art, I believe thattelecommunications technology can have a huge
impact on society and people’s minds in any period
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

and am encouraged every time I read that poem.
—Would you say a word or two to our young researchers?
As a researcher at NTT laboratories, I feel that
although the time spent on one research theme may
seem long, it is actually short. I really hope that bymaking the best use of the various intellectual fountains unique to NTT laboratories and using your time
well, you will strive to come across research themesthat only you can pursue. I believe that if you continue to make that effort, you will surely meet the
moment when you say, “This is it!” Since I was
assigned to NTT laboratories, I have delved into theresearch theme of optical communication systems,
which was a completely different field from my specialty as a student. At first, I sometimes got depressed
because I couldn’t participate in any discussions but I
continued trying and encountered challenging
research themes. When such an opportunity arises, I
want you to do your best to tackle the technical challenges that you will face.
I believe that one of the missions of researchers in
the field of system development for corporations is tocreate something useful for the world while balancing R&D and practical application. In particular,
R&D on communication systems cannot be done by
one person because of the wide variety of elemental
technologies. The happiest time for researchers iswhen they reach their goals. I’d like to experiencesuch times with outstanding partners and colleagues,
and I’d like all researchers who are taking up challenges in new fields to experience that happiness over
and over again. As I said before, technology concerning social infrastructure is critical, and its importanceis highlighted in times of emergency; however, it
usually goes unnoticed. It takes ten years for timeappropriate technological development to be put into
practical use. Although such development does notreceive much attention, it is important to continue it,
so I’ll do my best to do so.
References
[1]

[2]

Y. Miyamoto and R. Kawamura, “Space Division Multiplexing Optical Transmission Technology to Support the Evolution of Highcapacity Optical Transport Networks,” NTT Technical Review, Vol.
15, No. 6, 2017.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201706fa1.html
Y. Miyamoto, A. Sano, E. Yoshida, and T. Sakano, “Ultrahigh-capacity Digital Coherent Optical Transmission Technology,” NTT Technical Review, Vol. 9, No. 8, 2011.

11

Front-line Researchers

[3]

[4]

https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201108fa2.html
Y. Kisaka, M. Tomizawa, and Y. Miyamoto, “Digital Signal Processor(DSP) for Beyond 100G Optical Transport,” NTT Technical Review,Vol. 14, No. 9, 2016.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201609fa2.html
T. Kobayashi, F. Hamaoka, M. Nakamura, H. Yamazaki, M. Nagatani,
and Y. Miyamoto, “Ultrahigh-speed Optical Communications Technology Combining Digital Signal Processing and Circuit Technology,” NTT Technical Review, Vol. 17, No. 5, pp. 12–19, 2019.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201905fa2.html

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Interviewee profile
Yutaka Miyamoto
NTT Fellow, Director, Innovative Photonic
Network Research Center, NTT Network
Innovation Laboratories.
He received a B.E. and M.E. in electrical engineering from Waseda University, Tokyo, in 1986
and 1988, and a Dr. Eng. from the University ofTokyo. He joined NTT Transmission Systems
Laboratories in 1988, where he engaged in R&D
of high-speed optical communications systems
including the first 10-Gbit/s terrestrial opticaltransmission system (FA-10G) using erbiumdoped optical fiber amplifier (EDFA) inline
repeaters. He was with NTT Electronics
Technology Corporation between 1995 and 1997,
where he engaged in the planning and productdevelopment of high-speed optical modules atdata rates of 10 Gbit/s and beyond. Since 1997, hehas been with NTT Network Innovation Labs,
where he has been researching and developing
optical transport technologies based on
40/100/400-Gbit/s channels and beyond. He has
been investigating and promoting scalable opticaltransport networks with Pbit/s-class capacity
based on innovative optical transport technologies such as digital signal processing, space division multiplexing, and cutting-edge integrated
devices for photonic pre-processing. He is amember of the Institute of Electrical andElectronics Engineers (IEEE) and a Fellow of theInstitute of Electronics, Information andCommunication Engineers (IEICE).

12Feature Articles: Digital Twin Computing for Advanced
Interaction between the Real World and Cyberspace

Digital Twin Computing Initiative
Takao Nakamura
Abstract

Digital twins, which connect the real world with cyberspace, have been achieved through advances in
the Internet of Things and other information and communication technologies. NTT announced its
Digital Twin Computing Initiative in June 2019 [1] for envisioning a future in which the digitalizationof all types of objects (things) and their fusion with the real world accelerates in parallel with the evolution of the network and computing environment. This article provides an overview of this initiative,
describes the scope of its application, and presents its architecture and main issues surrounding implementation.
Keywords: digital twin, cyberspace, virtual societies

1. What are digital twins?
Digital twins have been gathering attention as ameans of linking the real world with cyberspace
thanks to progress in information and communication
technology (ICT). A digital twin is a mapping processthat accurately represents in cyberspace the shape,
state, function, etc. of a real-world object (thing) suchas a manufacturing machine in a factory, airplane
engine, or automobile. The use of digital twins makes
it possible to analyze current conditions, make predictions, and simulate various scenarios with respect
to things in cyberspace. It also enables the benefits ofadvanced ICT to be fed back to things in the realworld in diverse ways, such as by using the results ofprocessing in cyberspace to intelligently control
those real-world things.
As digital twins are created for all types of things in
the real world, we can expect a growing demand forlarge-scale simulations through the recombinationand interaction of many and varied digital twins
beyond those conventionally used in industry. For
example, such simulations could start with units ofindustrial machines and reproducing manufacturinglines to an entire factory or even an entire supply
chain including logistics. They could also reproduce
an entire city by recombining buildings, roads, automobiles, residents, etc. However, conventional digital
twins are only created and used for specific purposes
on a field-by-field basis, so this type of unrestricted
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

recombination and interaction of things has been difficult. Regarding human digital twins, the focus has
been on reproduction in physical terms based, forexample, on physiological measurement data. As aresult, simulations related to social behavior, such as
human communication, have been difficult to conduct.
2. What is Digital Twin Computing?
Digital Twin Computing (DTC) that we envision isa major expansion of the conventional digital twinconcept. It is a new computing paradigm that enables
reproduction of the real world on a totally new scaleand level of accuracy by freely recombining various
types of digital twins and performing diverse operations on them. It can also enable mutual interactions
in cyberspace that include the inner state of humans,i.e., thoughts and decisions of individuals, above andbeyond the physical reproduction of the real world(Fig. 1).
A key feature of DTC is the ability to arrange avariety of digital twins in cyberspace and create asingle world, i.e., a virtual world, driven on the wholeby the mutual interaction among those digital twins.
To enable various types of digital twins to be freely
recombined and subjected to analysis, trials, and predictions, we will develop a common means of performing large-scale and complex interactions among
digital twins. In addition to digitalization of the real13

Feature Articles

Conventional digital twins

DTC
Autonomous
socialsystems

Digital twins used individually

Socialsimulation& design

Humanaugmentation

PolicymakingTraffic jam

Autonomous
driving

Medicaltreatment

Robotics

MobilityCity

Replication, fusion, exchange
DTC

Digitaltwins

Digitaltwins

ThingsHumansThingsHumansFig. 1. Concept of Digital Twin Computing (DTC).

world as it actually is, DTC will make it possible toconfigure a virtual society with environmental conditions different from the real world and with things
that do not exist in reality. This, in turn, will make itpossible to conduct experiments and trials for a worldthat does not really exist, as in the design of a futuresociety.
Another feature of DTC is digital twin operations
that enable digital properties, such as replication andprocessing, to be used to the fullest. These operations
can be used, for example, to replicate digital twins
any number of times and conduct a variety of trials or
to exchange or converge some of the constituent elements of different digital twins to create digital twins
having properties that do not exist in reality (derivative digital twins). Combining derivative digital twinsand constructing a virtual society can expand realworld functions and interaction.
Yet another key feature of DTC and a major challenge is the reproduction of the inner state of humans.
For example, with respect to social aspects such as
human behavior and communication, reproducing
and representing in cyberspace the thoughts and deciNTT Technical Review

Vol. 18 No. 9 Sept. 2020

sions of individuals should enable interaction basedon diversity arising from individuality (personality)
in contrast to interaction between individual digital
twins with no personality based solely on statisticallyrounded-off values. This feature is described in detailin the article “Challenges Facing Human Digital
Twin Computing and Its Future Prospects” [2] in thisissue.
With DTC, the plan is to create diverse virtual societies in which all types of things and people interact
in sophisticated ways beyond the constraints of thereal world and to extend and transcend the real world
by fusing together virtual societies. The aim is toexpand human possibilities by extending the range ofhuman activities as far as virtual societies and to create innovative services that could not have heretofore
been achieved such as social design and decisionmaking support for solving complex social problems
through large-scale simulations or future predictions.
3. Application areas and use cases
Application areas and use cases of DTC are shown14

Feature Articles

Scale andgranularity ofdigital twins
Deeper

PersonExpansion of abilities
Expand abilities to bolster
intelligent tasks and skills andcollaborate with digital twinsthat are behaving and growing
autonomously

Decision-making
Review a variety of personalpossibilities and futurescenarios for use in one’s own
decision-making

Group

Town

Use of human intelligencebeyond temporal/spatialconstraints
Combine diverse personalities
and expertise for immediate
consensus building andbrainstorming

Medical care and health
Use biosensing and lifestyleanalysis to achieve
personalized/predictive medicine
and behavior modification

City

Nation

Globe

Space

Broader

Optimal control ofpeople/traffic flow

Future urban design

Prediction of urban energy
demandAchieve real-time demandprediction and highly efficientuse of energy and infrastructure(electricity, gas, water, etc.) at
the city level

Digital reproductions on a
global scale
Digitalize the topography,
climatic fluctuations, etc. of theentire Earth, predict and deal
with large-scale natural disasters,
and achieve sustainable national
and urban development

Conduct urban development andControl people/traffic flow using planning by predicting mediumspatial/temporal 4D information, and long-term changes in 4D
eliminate congestion and traffic information and achieve newjams in conjunction with weather urban design through trials andpredictions that include humanand schedules, and achieve asocial behavior
low carbon society

Fig. 2. Application areas of DTC.

in Fig. 2. In DTC, digital twins of diverse scale andgranularity can be taken up from a deep micro level,such as a single person, to a broad macro level, suchas a human group, city, nation, and even the entire
globe. Therefore, the following use cases in a wide
range of applications can be considered.
• Digitalize advanced decision-making processes
and expert skills and feed results back to real
people and things to reduce personnel training
costs and deal with labor shortages• Hold conversations with multiple future selvesthat simultaneously lead multiple lives to provide detailed support for making choices in one’s
real life• H old high-speed debates among digitalized
experts in various fields to quickly derive
responses to disasters, accidents, crime, etc.
according to current conditions• Control people/traffic flow using spatial/temporal fourth-dimensional (4D) information toeliminate congestion and traffic jams in conjuncNTT Technical Review

Vol. 18 No. 9 Sept. 2020tion with weather and schedules and achieve a
low-carbon society• Conduct trials and make predictions of mediumand long-term social activities in a city and simulate the views of future inhabitants then feed theresults back to urban designers to plan appropriate urban development• Digitalize the topography, climatic fluctuations,etc. of the entire Earth and predict and deal withlarge-scale natural disasters to achieve sustainable national and urban development
4. Architecture and main issues
surrounding implementation
DTC generates digital twins through the sensing ofreal-world things and humans, generates derivative
digital twins through digital twin operations, andconstructs virtual societies that recombine them thenfeeds the results of those virtual societies back to thereal world. The layered structure shown in Fig. 3 and15

Feature Articles

Use of human
intelligence beyond
temporal/spatialconstraints

Expansion ofabilities,
decision-making

Future urban
design/control

Simulations on
a global scale

Application layer

1

Execute applications using the digital-world-presentation layer

2

Structure, representation, andstandardization of digital twins– Structural definition of digital twins– Standardization of representationand interface– Modeling of human digital twins

Large-scale, highly accurate
future predictions through digital
twins of things and humans– Definition of virtual society models– Mechanism for driving virtualsocieties through temporal/spatial
sharing and autonomous operationof digital twins– Social simulations of humans– Highly accurate simulations,predictions, decision-making, andoptimization algorithms flexible toexternal policies (values, ethics,
etc.)

Human
groupsUrban
spacesTraffic
environments

Digital-world-presentation layer
Digital-twin usage frameworks
3
Real-world sensing/capturing– Highly accurate sensing/capturingof indoor/outdoor space and things– Advanced human sensing
Interaction between digital twins
and the real world

Digital-twin layer
Generating and maintaining digital twins of humans and things

4
DTC social research

Cyber/physical-interaction layer– Advanced feedback from digital
twins to real-world humans– Cyber/physical system security

Interaction between real space and cyberspace– Analyze social impact and issues
brought on by DTC progress– Technical/non-technical responses
to issues– Relationship between humans anddigitalization in the DTC era

HumansThingsReal space

Fig. 3. DTC architecture and main issues surrounding implementation.

summarized below is assumed with this DTC architecture.
• Real space: Consists of things and humans existing in the real world and the real space that
includes them
• Cyber/physical-interaction layer: Provides functions for collecting essential data to generate
digital twins through the sensing of things andhumans in real space and functions for feeding
the results of trials and predictions in virtualsocieties back to the real space• Digital-twin layer: Stores collected data, models,etc. and generates and stores digital twins using
those data
• D igital-world-presentation layer: Generates
derivative digital twins by performing operations(replication, fusion, and exchange) on digital
twins stored on the digital-twin layer and creates
virtual societies by combining derivative digitaltwins• A pplication layer: Implements and executes
applications using the digital-world-presentation
layer
The following summarizes the main issues thatneed to be addressed to achieve DTC including theabove use cases in accordance with this architecture.
NTT Technical Review

Vol. 18 No. 9 Sept. 2020(1) S tructure, representation, and standardization ofdigital twins
On the digital-twin layer, there is a need for a structural definition, representation format, and standard
interface for digital twins as frameworks for mutualinteraction among digital twins, which is the aim with
DTC. In addition, the constituent elements of humandigital twins including reproduction of a person’s
inner state and individuality must be defined andtechnology for modeling and reproducing humans
from sensing data must be established.
(2)	Large-scale, highly accurate future predictions
through digital twins of things and humans
On the digital-world-presentation layer, there is aneed for a model definition of virtual societies as aplace that recombines various types of digital twinsand for a mechanism that can drive virtual societies
based on temporal/spatial sharing and mutual interaction among those digital twins and on the autonomous operation of individual digital twins. Social
simulation methods targeting human communication,such as conversation and consultation, as well asgroup behavior must also be established based on
reproducing the inner state of humans. Based on all
the above, there will be a need to establish highly
accurate simulations, predictions, decision-making,16

Feature Articles

and optimization algorithms that can derive appropriate calculation results flexible to external policies onsocial norms, values, and ethics. These needs aredescribed in the article “Digital Twin Computing ofThings Opens Up a New Society” [3] in this issue.
(3) Real-world sensing/capturing and interaction
between digital twins and the real world
On the cyber/physical interaction layer, there will
be a need for highly accurate sensing and capturing of
not only of spaces and static subjects, such as outdoor
buildings, roads, and indoor structures, but also ofdynamic subjects such as moving cars. To enable ahuman digital twin to reproduce the user’s inner stateand individuality, there will also be a need foradvanced human sensing technology that can integrate measurement data from wearable devices, cameras, and microphones and behavior data on theInternet, etc. There will also be a need for feedback
methods using advanced human-machine interaction
devices to feed experiences in cyberspace back toreal-world humans and to enable activities in virtualsocieties via digital twins and expansion of realworld abilities through the help of cyberspace. These
needs are described in the article “Approaches to
Cyber-physical Interactions Linking the Real Worldand Cyberspace” [4] in this issue.
Finally, in the world of DTC that promotes thefusion of the real world and cyberspace, there will bean even greater need for cyber/physical system security.
(4) DTC social research
There will be a need to analyze the social impact
and issues brought on by progress in DTC, such asprivacy concerns, risk of digital crime, ethical concerns in relation to future predictions, free will, personal responsibility, and transformation of values in aborderless digital society, and to deal with theseissues in technical and non-technical ways. To gainsocial acceptance of DTC, cultural and sociological

NTT Technical Review

Vol. 18 No. 9 Sept. 2020studies must also be conducted on the new relationship between humans and digitalization. Please refer
the article “Social Issues with Digital Twin Computing” [5] in this issue for more details.
5. Conclusion
As described above, the issues that must be tackled
to implement the DTC initiative are extremely broadin scope. NTT laboratories aim to solve these issues
and implement the DTC initiative in society throughcollaboration with a wide range of experts in scienceand technology and diverse partners in industry. For
more details on the DTC initiative, please see theDigital Twin Computing white paper [6].
References
[1]

[2]

[3]

[4]

[5]

[6]

Press release issued by NTT, “NTT Proposes the ‘Digital Twin Computing Initiative’ – A Platform to Combine High-precision Digital
Information Reflecting the Real World to Synthesize Diverse Virtual
Worlds, Generate Novel Services and Bring about Society of the
Future,” June 10, 2019.
https://www.ntt.co.jp/news2019/1906e/190610a.html
I. Toshima, S. Kobashikawa, H. Noto, T. Kurahashi, K. Hirota, and S.
Ozawa, “Challenges Facing Human Digital Twin Computing and Its
Future Prospects,” NTT Technical Review, Vol. 18, No. 9, pp. 19–24,
2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202009fa2.html
M. Maruyoshi, M. Kawatani, and K. Mori, “Digital Twin Computingof Things Opens Up a New Society,” NTT Technical Review, Vol. 18,
No. 9, pp. 25–30, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202009fa3.html
Y. Koike, T. Matsubayashi, and S. Kondo, “Approaches to Cyberphysical Interactions Linking the Real World and Cyberspace,” NTT
Technical Review, Vol. 18, No. 9, pp. 31–35, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202009fa4.html
K. Takahashi, “Social Issues with Digital Twin Computing,” NTT
Technical Review, Vol. 18, No. 9, pp. 36–39, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202009fa5.html
White paper on Digital Twin Computing, https://www.ntt.co.jp/
svlab/e/DTC/whitepaper.html

17Feature Articles

Takao Nakamura

Senior Research Engineer, Supervisor, NTT
Digital Twin Computing Research Center.
He received a B.S. in mathematics fromWaseda University, Tokyo, in 1994, and a Ph.D.
in informatics from the Graduate University for
Advanced Studies, Kanagawa, in 2008. He
joined NTT Human Interface Laboratories in1994 and studied media processing technologies,
content distribution systems, artificial intelligence, and their applications. He was seconded
to the Ministry of Internal Affairs and Communications of Japan from 2007 to 2009, where he
was responsible for policy making and promoting national information security policies. He is
working on research and development of theconcepts and technologies for Digital Twin Computing.

NTT Technical Review

Vol. 18 No. 9 Sept. 202018

Feature Articles: Digital Twin Computing for Advanced
Interaction between the Real World and Cyberspace

Challenges Facing Human Digital
Twin Computing and Its Future
Prospects
Iwaki Toshima, Satoshi Kobashikawa, Hajime Noto,
Takao Kurahashi, Keiichi Hirota, and Shiro Ozawa
Abstract

Human Digital Twin Computing (DTC) aims to digitally express not only a person’s outer self, suchas physical characteristics, but also that person’s inner self such as personality and thoughts. We believethat digitizing information that includes the inner self can create unprecedented value. Collective consensus building, creating empathy and understanding of others, and future prediction and growth support
for individuals and societies are described as characteristic use cases of human DTC in this article. The
direction of future human-DTC projects, such as challenges in spreading human DTC throughout society, is also described.
Keywords: things DTC, human DTC, spatial DTC

1. Introduction
A human digital twin makes it possible to expand
the range of human activities from the real world tocyberspace. Interaction between oneself in the realworld and all digital twins in cyberspace is done
through one’s digital twin. Moreover, feeding backthe results of that interaction to oneself in the realworld makes it possible to use the experience gained
from activities in cyberspace in the real world(Fig. 1). The aim of human Digital Twin Computing(DTC) is to not only digitally represent a person’souter self in terms of, for example, physical andphysiological characteristics but also the person’s
inner self, for example, personality and thoughts.
A human DTC model that reproduces the individuality and characteristics of humans (for example, apersonality/thinking model that models behavioral
tendency, personality, and values or an ability modelthat models perception, knowledge, language ability,
physical ability, etc.) defines the behavior of a digital
twin. By reacting to other people’s actions in cyberspace as if they were our real-world selves and by
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

making them behave autonomously in a virtual society, human digital twins can engage with others as
our real-life selves (Fig. 2). Human DTC is defined asa whole system that develops and enables humandigital twins.
In this article, the advantages that human DTC provides are first summarized, then specific use cases
demonstrating these advantages are explained. Next,
the unavoidable considerations necessary for human
DTC to be accepted in society are summarized, andthe long-term prospects of human-DTC projects aredescribed after that.
2. Value of human DTC
A human digital twin is not only composed of data
showing your state and behavior, it is also composedof a model that expresses your individuality and emotions such as your tendencies regarding judgment andbehavior. As a result, it enables you to interact withothers in a virtual society and carry out autonomous
activities there as if it were yourself. We believe that
this interaction essentially has three advantages. The
19

Feature Articles

Person-expanding activities fromthe real world to cyberspace
Digitaltwin of that
personPerson in
the real
world

Human
interaction

Interaction
with things

Use of services

Digitaltwin of that
personPerson in
the real
world

Digitaltwin ofthings

Real-worldthingsDigitalservices

Cyberspace

Fig. 1. Human interaction in cyberspace via a digital twin.

D
D

D
D

D
D

D
D

D

Past

D
D

D
D

D
D

D

Futureprediction

D
D

D
D

D
D

D

Anonymization

Reproduction

D
D

D
D

D

Derivative
digital twin

Expansion

FusionExchange

Personality/
thinking model

Ability modelSocialinformation

Values, personality,
behavioral
tendencies, etc.

Perception, physicalability, knowledge,
language, etc.

Address,
occupation,
relationships, etc.

Physiologicalinformation

Psychological/
behavior
information

Exercise/
activity information

Environmental
information

Electroencephalogram,heartbeat, sweating,etc.

Emotions, tension,
speech, attitudes,
etc.

Physical movement,
behavior, movement,
etc.

Place, temperature,situation, clothes,
etc.

Human
sensing

Current (original)
digital twin(constantly updated)

Action on
humansFig. 2. Human digital twin created using DTC.

NTT Technical Review

Vol. 18 No. 9 Sept. 202020

Feature Articles

Seniorcolleague

Relateddepartment
staffManager

Digital twin that
reproduces the group

Digital twin that
reproduces the individual

Good

Fig. 3. Group consensus building with human digital twins.

first is being able to be used for various tasks that are
beyond our physical constraints. Regarding communication in cyberspace, a huge amount of processing
can be performed at high speed beyond the real-timeprocessing performed during communication between
real people. The second advantage is that having amodel that expresses emotions makes it possible totransmit various emotions as they are. In this virtualsociety, you can convey nuances, so you can create aworld in which your thoughts do not have discrepancies. The third advantage is being able to interact withpeople on the basis of social aspects and diversity,and this ability will make it possible to (i) precisely
analyze those interactions while taking into accountthe individuality of humans and (ii) predict the future.
It is an advantage that can be taken in various ways,
such as for consultations aimed at changing a person’s behavior. The source of these advantages lies in
the implementation of human personality and thecreation of models of emotions. We will promoteresearch and development by treating this humanindividuality as one of the most important issues
regarding human DTC.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

3. Use cases of human DTC
3.1 Collective consensus building
A human digital twin has a memory of personal
knowledge, experience, etc., so it can think and judge
while having the same personality and sense of values
as the real-life person and engage in various tasks(Fig. 3). Regarding DTC, a digital twin replicatescommunication that takes place in the real world, so
it can engage in advanced tasks that require communication with multiple people. A typical example isconsensus building during a meeting.
In this use case, it is possible to simulate the thinking of a group consisting of multiple digital twins andvisualize it in a timely manner without involving
people in the real world. Moreover, the tacit knowledge inside the individual can be drawn out by working with a digital twin of an expert who is not there.
As a result, it is expected that the group will be able
to smoothly facilitate the emergence of new ideas anddecision-making. Moreover, through reproduction ofa digital twin, digital twins can exist in multiple virtual societies at the same time regardless of the stateof the real world. Therefore, we believe that in thereal world, it will be possible to instantly form a consensus among a large number of people, namely, a21

Feature Articles

task that has been impossible until now due to timeand space constraints.
3.2 C reating empathy and understanding of others
Digitization of sensations and emotions is also an
important element in creating a human digital twin.
By using DTC to convert the senses and emotions
reconstructed from a person’s scan information as adigital twin into a form that another person can understand and feeding them back to the real world, it ispossible to create an empathetic state that is understandable to each party (Fig. 4). An example in whichhaving empathy using sensations and emotions isuseful is a diagnosis of a patient given by a doctor inan examination room. In a real-life diagnosis, thepatient can only qualitatively convey to the doctor
verbal feelings such as pain and anxiety. Even if pain
can be quantified, the sensations and emotions that anindividual feels will vary from person to person. As
such, it is difficult to determine if the patient
explained his/her symptoms correctly and if the doctor’s explanation is correctly transmitted to thepatient. In a diagnosis through human DTC, the digitized sensations of pain and anxiety are converted
into a form that can be transmitted to the other party
then fed back to the real world to enable effective
communication with the other party. Creating empathy by using a person’s digital twin will revolutionize
communication, that is, it will be possible to create aworld in which a person can not only tell the otherparty what he/she wants to say in words but also makethe other party understand what he/she cannot say inwords.
3.3 Future prediction and growth support
For a human digital twin created using DTC, it isalso possible to use micro-level simulation results forindividuals to determine their behaviors (Fig. 5). It is
expected that using derivative digital twins with newknowledge and ability on the basis of fusion andexchange will make it possible to predict other future
possibilities, such as what happens if a person
acquires new knowledge, and use such predictions togrow society and oneself. An example of growing
oneself is learning a new language. Self-growth has
its own motivation and goal setting and is gained
through experience through physical and intellectual
activities. With a human digital twin, you can create
multiple derivative digital twins of you and simulate
all the languages you want to learn by using each ofyour derivative digital twins. Through exchanging
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Sensations and emotions are modeled
and restored as a digital twin fromscan information of the person.

DTC

Sensations andemotions are fed
back to real world.

Human DTC transforms humansensations and emotions into a form thatcan be transmitted to the other person.

Fig. 4. S haring sensations and emotions through a humandigital twin.

with other digital twins in English, German, French,
etc. in a virtual society, you can learn these languages
in a practical manner and receive guidance. You canalso feel emotions and excitement as if you actuallyexperienced the interaction. Based on the simulation
results, new knowledge about languages that are easy
for you to learn, countries that impressed more, etc.
are fed back to the real world to promote and support
your growth.
4. Social acceptance and security ofhuman DTC
As mentioned above, human DTC is expected to
expand people’s abilities and produce beneficial
results. However, for people to receive these benefits,
the social acceptance of a worldview through human
DTC must be fully considered. It is essential to raise
people’s awareness and individual acceptance concerning, for example, (i) psychological resistance tothe creation of human digital twins with the same
appearance and thoughts as oneself and (ii) privacy
concerns about the aggregation of various types of
information about oneself. Also, a human digital twin
may engage in the same economic activities as thehuman it represents, and that situation may create
new social and/or legal systems. The social and22

Feature Articles

In addition to learning
language, emotions
experienced with cultural
exchange can be predicted.
Human
DTC

Acquired knowledge
and impressions are
fed back to real world.

Multiple derivative digital
twins are generated.

Fig. 5. Future prediction and growth support through human digital twins.

economic effects of human digital twins doublinghuman knowledge and abilities and multiplying theactivities of people many times will cause a large
paradigm shift. Concerning the various effects thatcreation and popularization of human DTC will haveon the real world, it is necessary to search for the ideal
form of human DTC while discussing those effects
with experts in fields such as the humanities (philosophy, ethics, etc.), law, and sociology (social psychology, behavioral economics, etc.).
Moreover, when it is assumed that human DTC andthings DTC reflect all the information in the realworld, it will become clear that various threats andrisks can arise from DTC. Accordingly, it is necessary
to thoroughly consider the security concerning bothtypes of DTC and handle them from various aspects,
regardless of whether those aspects are technical or
non-technical. With that necessity in mind, we will
introduce ethical codes and legal concepts and create
an orderly DTC world equivalent to the real world.
On the basis of our extensive knowledge gained from
activities concerning security technology and cybersecurity cultivated over many years, we will also create DTC security based on the idea of security bydesign and privacy by design.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

5. Concluding remarks
Envisioning some of the goals mentioned in thisarticle, research on human DTC has just begun. However, the base technology, for example, the technology for recognition and generation of speech, language, images, etc., has greatly evolved by using
artificial intelligence (AI). With respect to thesetechnologies, we want to accelerate research anddevelopment on human DTC by focusing on the individuality of people and delving further into people’s
inner self. Furthermore, from the perspectives ofindividuality and the inner self as well as permeation
into society, it is necessary to expand human DTC tofields other than AI and engineering, namely, biology
and medical (including brain science) and ethics,
philosophy, and the other humanities (such as behavioral economics). Our human-DTC research team is
not only pursuing research but also promoting technological implementation of human DTC by linking
research in these wide-ranging fields as a hub. We
believe that getting people to recognize the value thathuman DTC brings to the world will be essential inbringing about changes to accepted social norms.

23

Feature Articles

Iwaki Toshima

Senior Research Engineer, NTT Digital Twin
Computing Research Center and NTT MediaIntelligence Laboratories.
He received a B.E. and M.E. from the University of Tokyo in 2000 and 2002, and a Ph.D. from
Tokyo Institute of Technology in 2008. He was at
NTT Communication Science Laboratories from2002 to 2013, where he researched robot-audition and auditory psychophysics. He developed
NTT’s home-router and FTTH (fiber-to-thehome) services while at NTT WEST from 2013
to 2019. Since 2019, he has been researching
human digital twins at NTT Digital Twin Computing Research Center. He is a member of theInstitute of Electrical and Electronics Engineers(IEEE), Robotics Society of Japan, VR Societyof Japan, Acoustical Society of Japan (ASJ), andthe Japan Society of Mechanical Engineers.

Satoshi Kobashikawa

Senior Research Engineer, Social Knowledge
Processing Laboratory, NTT Media Intelligence
Laboratories.
He received a B.E., M.E., and Ph.D. from the
University of Tokyo in 2000, 2002, and 2013.
Since joining NTT in 2002, he has been researching speech recognition and spoken language
processing. He received the Kiyasu Special
Industrial Achievement Award from the Information Processing Society of Japan (IPSJ) in 2011,
the 58th Maejima Hisoka Award from the Tsushin­bunka Association in 2012, and the 54th Sato
Paper Award from ASJ in 2013. He is a memberof ASJ, IPSJ, and the Institute of Electronics,
Information and Communication Engineers(IEICE).

Hajime Noto

Senior Research Engineer, Universe Data Handling Laboratory and NTT Digital Twin Computing Research Center, NTT Media Intelligence
Laboratories.
He received a B.E. and M.E. in industrial engineering from Kansai University, Osaka, in 1997
and 1999. In 1999 he joined NTT Cyber Space
Laboratories (now, NTT Media Intelligence
Laboratories), where he researched three-dimensional (3D) input systems. He is currently
involved in research and development of humandigitization. He is a member of the Institute of
Image Information and Television Engineers(ITE).

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Takao Kurahashi

Senior Research Engineer, Cybernetic Intelligence Research Project, NTT Service Evolution
Laboratories and NTT Digital Twin Computing
Research Center, NTT Media Intelligence Laboratories.
He received an M.E. in industrial engineeringfrom Hosei University, Tokyo, in 2002 and joined
NTT the same year. His research interests
include human-computer interaction and robotics.

Keiichi Hirota

Senior Research Engineer, Secure Data Sharing Project, NTT Secure Platform Laboratories
and NTT Digital Twin Computing ResearchCenter, NTT Media Intelligence Laboratories.
He received a B.S. and M.S. from Mie University in 1995 and 1997, and a Ph.D. in informatics
from the Graduate University of Advanced Study(SOKENDAI), Kanagawa, in 2008. He joined
NTT in 1997. His current research interests are
security and privacy in information processing,information sharing, and data utilization. He is amember of IPSJ.

Shiro Ozawa

Senior Research Engineer, Supervisor, NTT
Digital Twin Computing Research Center and
NTT Media Intelligence Laboratories.
He received a B.E. and M.E. from Tokyo University of Mercantile Marine (now, Tokyo University of Marine Science and Technology) in1997 and 1999. After joining NTT as a researcher, he worked for NTT Cyber Space Laboratories(now, NTT Media Intelligence Laboratories). His
fields of interest are computer vision, video communication, user interfaces, human-computerinteraction, and 3D displays. He is a member of
IEICE, ITE, and the Institute of Image Electronics Engineers of Japan.

24

Feature Articles: Digital Twin Computing for Advanced
Interaction between the Real World and Cyberspace

Digital Twin Computing of Things
Opens Up a New Society
Masahiro Maruyoshi, Muneyuki Kawatani,
and Koya Mori
Abstract

Through Digital Twin Computing (DTC), we aim to enhance the quality of people’s lives by digitizing
objects as well as actual humans with high accuracy and enabling the digitized objects to be used by real
people. To achieve DTC, we are developing (i) digital-twin creation technology that enables interaction
among people and digitized objects and (ii) large-scale, high-resolution, and highly accurate technology
for simulating the flows of those digitized objects and people. Our efforts concerning these technologies
are described in this article.
Keywords: digital space, simulation, smart city

1. Introduction
At Digital Twin Computing Research Center, weare developing digital-twin creation technology forreproducing the real world and large-scale real-time
simulation technology.
Digital-twin creation technology for reproducing
the real world aims to digitize the physical shape andstate of objects and create a space (digital-twin space)in which real people can share and operate these
digitized objects via real space. Three-dimensional(3D) modeling technology is used in construction andmanufacturing for digitizing an object into 3D data,
and we will forge ahead with creating a digital-twin
space by integrating this and other technologies as
elemental technologies.
Large-scale real-time simulation technology will
make it possible to accurately predict people flow,
traffic flow, weather conditions, energy flow, etc.,
which have been digitally reproduced, over a wide
area such as a city or whole country in real time. We
aim to use this technology for real-time control of city
functions and evaluating the impact on urban life due
to improvements in urban functions. This article
describes current technological trends and issues
related to digital-twin creation technology for reproducing the real world and current efforts concerning
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

large-scale real-time simulation technology.
2. Digital-twin creation technology forreproducing the real world
When people operate digitized objects, various
devices, such as a mouse, keyboard, or touch panel,
have been used. We believe that it will be more
important to interact with digitized objects moreintuitively and directly by using vision, hearing, andtouch. In addition to previously proposed hardware
technology for wearable devices such as head-mounted displays and virtual reality glasses, 3D modelingtechnology for digitizing physical objects in 3D will
also be important. Such technology can be classified
into the following three categories on the basis of thesize and movability (portability) of an object.
(1)	Geospatial digitization technology (for infinitely extending objects with no portability)(2)	Technology for digitizing buildings (for huge
objects with no portability)(3)	Technology for digitizing things (for objectsof various sizes with portability)
The following technological trends are explained inline with these categories.

25

Feature Articles

Building data of Autodesk Revit (a BIM tool) [1] (*CC BY-NC-SA 3.0) are placed on the map by using Esri
ArcGIS [2] (a GIS tool).
It is thus possible to construct a city model with an accurate shape, but the model must be manuallyaligned for precisely placing the data.

Fig. 1. Example of creating a city model based on a 3D model using GIS and BIM.

2.1 G
 eospatial digitization technology for infinitely extending objects with no portability
A geographic information system (GIS) has beenused for digitizing geospatial information about wide
areas, such as cities, topographical information, andadditional information in an integrated manner. Topographical information is digitized by extracting
shapes from aerial radar and satellite images. In addition to this topographical information based on thecoordinate system using latitude, longitude, and altitude, layers are prepared for each analysis target(such as a plane map and its topographical information, borders, and features), and they can be overlaid.
These types of information are suited for representation in a macroscopic perspective because they havethe number of characteristic points and degree of
accuracy with which a wide range of information canbe drawn at a practical speed.
2.2 T echnology for digitizing buildings for huge
objects with no portability
Building-information modeling (BIM) and construction-information modeling (CIM) have beenused for designing the structures of features, such asbuildings, roads, and bridges, as 3D models. BIM/
CIM also manages attributes such as materials usedin buildings and their strength. For constructing a realbuilding, this technology is excellent at representing
a microscopic perspective because of its high accuNTT Technical Review

Vol. 18 No. 9 Sept. 2020

racy. It is also possible to represent a city by placing
3D models created with BIM on a GIS (Fig. 1), andthere has been a trend toward making such a model
available—together with the GIS—as an open-citymodel [1, 2].
2.3 T echnology for digitizing things for objects ofvarious sizes with portability
Computer-aided design (CAD) is used to design the
structures of various industrial products as 3D models. Using CAD makes it possible to check interferences between objects and measure strain using
physical simulation before manufacturing an actual
product. For manufacturing real objects, the technology is excellent at representing a microscopic perspective due to its extremely high accuracy. However,the data format differs across various industrial fields
and is not standardized.
Photogrammetry technology [3] (Fig. 2), which
estimates feature points from moving images ofobjects and creates 3D models from these points, andshape-extraction technology by radar measurement
have been proposed [4]. While there is an advantage
in that all objects can be modeled in 3D from images,
the extraction accuracy depends on the state of theenvironment, such as lighting level, and there are still
many challenges with real-time extraction of moving
objects in video processing.

26

Feature Articles

Input image(More than 100 images with different positions/directions)

Extracted 3D model(Parts with strong specular reflection have not been extracted.)

Fig. 2. Example of extracting a 3D model by using a tool (Meshroom) implemented with photogrammetry technology.

3. Challenges towards digital-twin creation
technology for reproducing the real world
Digital Twin Computing (DTC) technologies will
make it possible for a real person to interact in a virtual space with various objects in the real world bydigitizing all real objects. An interaction in a virtual
space is considered to have meanings ranging from,for example, feedback from the senses (e.g., feeling asensation when touching something) to acquiring
information (such as seeing the usage history whentouching something). For the 3D-modeling technology mentioned above, attributes are defined according to the usage in each industrial field. However,they are not defined under the assumption that digitized objects interact with real people.
To create a digital-twin space in which digitizedobjects can move dynamically in real time, the digitized objects must be managed in an integrated manner. For example, objects digitized by a third party
can be placed in a virtual space where buildings are
geographically spread out like a city and many people
can touch them at the same time. To do this, different
attributes (3D coordinate system, accuracy, data format, etc.) created in various industrial fields must be
standardized. We aim to form—as one framework—(i) attributes that enable people to interact with digital
twins of things and (ii) interface conditions for handling digitized objects generated in multiple industrial fields in a virtual space.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

4. Large-scale real-time simulation technology
Large-scale real-time simulation technology, namely, simulation using digital twins constituting a digitized city space, aims to enable real-time optimal
control of city-wide activities (such as people flow,
traffic flow, logistics, and energy flow) and mid- tolong-term planning and decision-making in cities andgovernments. The key features of this DTC simulation is that it manipulates wide areas and achieves
high accuracy. The challenges and efforts concerning
each key feature are described below.
4.1 Wide-area simulation
Simulation of a city includes simulation of people
living in the city and their activities, transportation(cars, etc.), and environment (weather, etc.). Tokyo istaken as an example city. Central Tokyo has a population of over 13 million and over 3.5 million vehicles
on its streets [5, 6]. Since it is not realistic to conduct
these simulations on a single computer, it is possible
to, for example, divide Tokyo into several areas, andsubject each area to parallel distributed processing.
However, in this case, it is necessary to consider
people and vehicles moving across these areas ofparallel distributed processing (Fig. 3). We areresearching wide-area simulation with which movement across such areas is assumed.
4.2 Highly accurate simulation
When simulations are used for real-time optimal
control of activities of an entire city and for planning
27

Feature Articles

Single area calculated by parallel distributed processing
Trajectory of the route that the vehicle travels
The point where vehicles cross over areas of parallel distributed processing

Fig. 3. Example of a vehicle moving across areas of parallel distributed processing.

and decision-making by cities and governments, it isessential to improve the accuracy of the simulation
results. Therefore, it is possible to improve simulation accuracy by including computational elementsthat have not been considered in the simulation thus
far, for example, wind flow passing between buildings, a building’s exposure to sunlight, activity ofeach citizen, and movement of each car. However, if
the number of such calculation elements is simply
increased, the calculation amount will increase exponentially; therefore, we are clarifying which calculation elements have a significant effect on the simulation results and studying how to incorporate theminto the simulation.
It is also necessary to consider fluctuations in simulation results caused by uncertain phenomena thatcannot be easily predicted from past events, such as
traffic accidents and heavy rain. To execute a simulation considering these uncertain events, it is necessary to not only execute numerical analysis on acomputer but also reflect the events that occurred in

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

the real world in the simulation calculation in realtime to bring the calculation results closer to theevents in the real world. We are currently researching
technology to satisfy these requirements (Fig. 4).
Through these efforts, we aim to enable large-scale,
high-resolution, and highly accurate city simulations,which will be useful for optimizing city activities andhuman decision-making.
5. Future prospects
We will continue researching and developing (i)digital-twin creation technology for reproducing thereal world that enables objects and spaces to interactand (ii) technology that simulates the future in a digital space and feeds the prediction results back to thereal world. While pursuing these efforts, we will also
focus on implementing the 4D digital platformTM [7].
To achieve DTC, we will also promote a partnering
strategy with various companies across industries.

28

Feature Articles

Collects real-world weather, temperature,
water temperature, etc. via sensors

Performs highly accurate simulations
using collected information

Collects vehicle-position andspeed information from fixed-point
and vehicle-mounted sensors

Large-scale real-time
simulation platform

Fig. 4. Simulation using phenomena in the real world.

References
[1]

[2]
[3]
[4]

Website of Autodesk, Revit products, https://knowledge.autodesk.
com/support/revit-products/getting-started/caas/CloudHelp/
cloudhelp/2018/ENU/Revit-GetStarted/files/GUID-7B9C7A691083-406D-A01F-53D405C167F3-htm.html
Website of Esri, ArcGIS, https://www.esri.com/en-us/arcgis/aboutarcgis/overview
Website of AliceVision, https://alicevision.org/
D. Zou, P. Tan, and W. Yu, “Collaborative Visual SLAM for Multiple
Agents: A Brief Survey,” Virtual Reality & Intelligent Hardware, Vol.
1, No. 5, pp. 461–482, 2019.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

[5]
[6]

[7]

Website of Tokyo Metropolitan Government, Statistics of Tokyo,
https://www.toukei.metro.tokyo.lg.jp/homepage/ENGLISH.htm
Automobile Inspection & Registration Information Association,“Number of Vehicles Owned by Prefecture and Type of Vehicle” (inJapanese).
https://www.airia.or.jp/publish/statistics/ub83el00000000woatt/01_2.pdf
Press release issued by NTT, “4D Digital Platform™”; Integrates
Various Sensing Data in Real Time and Enables Future Predictions,”
Mar. 26, 2020.
https://www.ntt.co.jp/news2020/2003e/200326c.html

29

Feature Articles

Masahiro Maruyoshi

Senior Research Engineer, NTT Digital Twin
Computing Research Center, NTT Media Intelligence Laboratories.
He received a B.E. and M.S. in informationengineering from Nagoya University, Aichi, in
2000 and 2002. He joined NTT in 2002 and wasengaged in research and development of software
defined network technologies. Since 2020, he has
been in charge of research for Digital Twin Computing at NTT Digital Twin Computing ResearchCenter.

Koya Mori

Senior Research Engineer, NTT Digital Twin
Computing Research Center.
Since joining NTT in 2004, he has been
involved in the research and development of an
IoT application based on software technologies,
such as OSGi, OpenStack, and Edge Computing.
Since 2020, he has been in charge of research for
Digital Twin Computing at NTT Digital Twin
Computing Research Center.

Muneyuki Kawatani

Senior Research Engineer, NTT Digital Twin
Computing Research Center, NTT Media Intelligence Laboratories.
He received a B.E. and M.E. from Kobe University in 2004 and 2006. He joined NTT in 2006
and was engaged in research and development ofapplication platform systems for open source
software. Since 2020, he has been engaged in the
Digital Twin Computing project, specificallydeveloping the simulation technology describedin this article. He also worked at NTT DOCOMO
from 2011 to 2016 and developed a network
system for smartphones.

NTT Technical Review

Vol. 18 No. 9 Sept. 202030

Feature Articles: Digital Twin Computing for Advanced
Interaction between the Real World and Cyberspace

Approaches to Cyber-physicalInteractions Linking the Real Worldand Cyberspace
Yukio Koike, Tatsushi Matsubayashi,and Shigekuni Kondo
Abstract

From among the technologies incorporated in the cyber-physical interaction layer connecting the realworld and cyberspace towards achieving Digital Twin Computing, this article introduces biologicalsignal-decoding and feedback technologies to extend motor-function capabilities by feeding the resultsof operations done with a person’s digital twin back to him/herself and zero latency media technologythat enables operations that do not feel uncomfortable by feeding back a prediction presentation thateliminates delay and overcoming the physical distance of the real world and human perception of time.
Keywords: cyber-physical interaction, zero latency, biological signal decoding and feedback

1. Cyber-physical interactions linking the realworld and cyberspace
NTT laboratories are aiming to actualize Digital
Twin Computing (DTC) to create diverse virtual societies through interactions between various objects
and humans at a higher level beyond the constraintsof the real world and to expand the scope and potential of human activities by fusing virtual societies
with the real world. From among the technologies
incorporated in the cyber-physical interaction layer
connecting the real world and cyberspace, this article
introduces biological-signal-decoding and feedback
technologies to extend motor-function capabilities by
feeding the results of operations done with a person’s
digital twin back to him/herself and zero latency
media technology to have these operations be discomfort-free through feedback of near-future prediction results (Fig. 1).
2. Biological-signal-decoding and feedback
technologies to make DTC possible
One aspect of a human digital twin in DTC is that it
NTT Technical Review

Vol. 18 No. 9 Sept. 2020is an interface for one’s real-world self to interact
with various digital twins in cyberspace or interact
with other real-world humans. Specifically, a digitaltwin enables experiences in cyberspace to be conveyed to one’s real-world self via a human-machineinterface (HMI). Thus, an HMI plays an importantrole in the interactions between digital twins and thereal world [1]. Digital twins of humans are not justhuman reproductions but also expand capabilities
through exchange and fusion in cyberspace, which
are key digital twin operations [1]. An HMI also plays
an important role in enabling a human to demonstratesuch capabilities in the real world.
Therefore, we have embarked on HMI-technology(biological-signal-decoding* and feedback technologies) initiatives to achieve this from the perspective of
expanded motor-function capabilities, which are particularly important elements of expanded humancapabilities in both cyberspace and the real world.
In general, motor control in the human body results
when stimulation received by a sensory organ is*

Decoding: Analysis and understanding of biological signals (sensory reproduction from brain waves, etc.).

31

Feature Articles

Abilities

Signals

Information

Abilities

Signals

Information

Operation functions of exchange andfusion of physical abilities andbiological signals
Technology to achieve digitaltwin operations without
discomfort through the use ofhuman senses and near-future
prediction results

Mediation functions
Physicalabilities

Biologicalsignals

Physicalabilities

Sensor information

Biologicalsignals

Biological signal adjustment
and conversion technology todetect and absorb inconsistencies
and gaps (individual differences,etc.) and achieve optimal physicalmotion (mediation functions)Sensor information

Create digital twin
Clarification of the relationshipbetween biological signals(electroencephalogram,
electromyogram, etc.) andhuman motion control(modeling, understanding
current status and maximum
values)Zero latency functions(senses, prediction)
Decoding functions(sensing, modeling)
Real-world human

Feedback functions(perception, motion support)

Technology to reproduce
motions/sensations and make thebody perceive them and motorcontrol technologies including
physical functional enhancements(exoskeletons, etc.)

Real-world humanFig. 1. Technology supporting the cyber-physical interaction layer.

perceived by the brain via the nervous system, cognitive activity such as planning for action occurs in thebrain, and commands for the movements to be executed are transmitted via the nervous system to themusculoskeletal system where the muscles thatreceived the signals contract [2]. The exchange andfusion between digital twins regarding sensory functions, nervous functions, musculoskeletal functions,the biological signals transmitted between these functions, and feedback of the results of these to humans
in the real world to enhance functionality and signal
distribution is our objective for enhanced motorfunction capabilities. This initiative will make it possible for humans in the real world to acquire enhanced
motor-function capabilities through physical support
or training in which the motor-control functions andbiological signals of a skilled person are projected
onto a trainee.
To feed expanded motor-function capabilities backto humans in the real world to enhance human motor
functions, we believe the following biological-signaldecoding and feedback technologies are important.
• Biological-signal-decoding technology: Clarifying and digitizing the relationships between
biological signals (electroencephalogram, electromyogram, etc.) and movements between thefunctional elements of human motor control(digital informatization through decoding andextraction and patterning of time-space feature
values for highly reproducible sense and movement signals)• Feedback technology: Technology to reproduce
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

movements and sensations and make the bodyperceive them and motor-control technologies
including expanded physical functionality (foruse and control of exoskeletons)• Mediation technology (related to feedback technology): Biological signal adjustment and conversion technology (mediation functions) todetect and absorb inconsistencies to optimize
body movements based on decoded information
In particular, we consider the role of the mediation
functions to be important in expanding capabilities
through the digital twin operations of exchange andfusion. Exchanging and fusing the functions and signals of human motor control between digital twins
will not only involve simple feedback to the realworld but will also require detection and adjustment
for inconsistencies between the real world and digitaltwin information expected to occur with such feedback. More specifically, it will be necessary to absorb
the individual differences in functionality and signaling and the differences in the states of digital twinsand humans in the real world. For example, if biological signals from a skilled athlete are fed back tosomeone who is unskilled, the unskilled person might
not be able to move as originally expected due to theirmuscle strength, limited range of movement of theirjoints, or their current posture. There are also safetyissues that may lead to ruptured muscle fibers, joint
injuries, etc. Therefore, we are aiming to develop
HMI technology that includes mediation functions inwhich the strength of signals to be fed back areadjusted or converted for movement more suitable to32

Feature Articles

the human to create a world in which people withextended capabilities move as if with their own body.
We are proceeding with research and developmentof these biological-signal-decoding and feedback
technologies to apply exchange and fusion, which arecentral to DTC, and extend human capabilities.
3. Zero latency media technology for DTC
In the real world, the concepts of space and time
can be considered closely linked as individual people
and objects are defined by unique spatial coordinates
and the time axis. Information exchanges are carriedout at the speed of light for visual information or the
speed of sound for auditory information. Therefore,
information transmission cannot physically exceed
the speed of light. For this reason, delays occur when
operating something remotely, which means physicalconstraints must be transcended to recognize theobject being operated as being in the same space.
Humans also experience cognitive delays with information transmission when signals are sent to thebrain from sensory organs. Recognition and judgment are then carried out, leading to movement, etc.
For this reason, the cognitive constraints of humanrecognition and judgment must be surpassed, even forevents that take place in the same space. By focusing
on physical latency that occurs with communications,etc. when operating objects and latency due to humancognition through DTC, we have embarked on thezero latency media technology initiative with a focus
on the perspectives of distant-independent expanded
capabilities that transcend the constraints of spatial
distance and individual-independent expanded capabilities that step into the world of cognition notdependent on personal abilities. Below, we give an
overview of these expanded capabilities and introduce the direction in which we would like to takethem.
3.1 Distant-independent expanded capabilities
In DTC, in which people and objects are fused andcoordinated when providing feedback to the realworld through the cyber-physical interaction layer,
the concept of space-time must be defined in the
cyberspace framework instead of the simple synchronous time progression in the digital-world presentation layer. There is always a physical communications delay when providing feedback to the realworld, which can cause problems with a range ofapplications. Examples include discomfort due to lipsync deviation with video streaming [3] or video
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

motion sickness that occurs due to delays in systemprocessing with virtual reality and augmented reality[4, 5]. These issues manifest as discomforts that
occur when sense-organ information different fromthe information expected by the human is given [4],which is also a major problem with DTC.
As well as feedback to the real world, there are evengreater delays in human-to-thing and human-tohuman interactions between remote locations, which
necessitate technology to align the two different timeaxes to be fused. For example, in principle, with signal processing to remotely operate a machine, predicting environmental changes occurring with respect
to the input makes alignment to the same time axis in
the same space-time in cyberspace possible. However, simple physical prediction alone does not lead to aworld where people and objects are fused. It is known
that perceptual discrepancies about operations occur
when people manipulate objects, but these can beeliminated with training and experience. Taking theexample of driving a car, different capabilities areused when driving at 50 km/h and when racing at 300
km/h to process the sensation of driving gained fromhandling the vehicle or the resistance between thetires and the road surface, and the cognitive sense oftime is completely different.
Hence, we are conducting research into modeling
how prediction is performed in the brain as well as
prediction of environmental changes by matching the
cognitive delays (called sensory delays) that arise
with human body movements for manipulating
objects with system problems that occur when operating objects and delays in remote communications(called physical delays). The aim is to expand capabilities to freely operate objects remotely by eliminating the discomfort that occurs due to these physicaland sensory delays and overcoming, through cyberspace, the physical gaps in distance in the real world.
3.2 I ndividual-independent expanded capabilities
Cognitive-delay tolerances and predictive abilities
that occur when humans manipulate objects vary
greatly from person to person. For example, there arelarge differences in the abilities of skilled workers,
such as surgeons, airplane pilots, car racers, or professional athletes, compared to those of untrained
humans, and it is possible to eliminate such differences in cognitive ability by experience and training.
Although acquiring some of these skills requires a
great deal of training, providing a simulated experience of the world experienced by skilled workers is
33

Feature Articles

also an effective way to learn. For example, learning
to pilot an aircraft or drive a racing car does not startout with maneuvering an actual machine but entails
sufficient training with a flight or drive simulator.
Similarly, training for competitive swimming is possible by having the experience of swimming fast by
being pulled by a rope, or baseball training to raise
one’s ability to respond to fastballs can be done bypracticing with high-speed balls from a pitchingmachine. There has also been research into expanded
capabilities that involves providing people with theexperience of improved reflexes by stimulating themuscles using electrical muscle stimulation [6],enabling responses faster than the human cognitive
delay. This research showed that it is possible to vary
the amount of stimulation to give participants the illusion that they are actually performing a behavior suchas grabbing a falling stick or taking a picture of a flying ball.
In light of this, we will study the human ability tomake predictions by researching both human physical-reflection mechanisms and prediction mechanisms by modeling the speculative decision-making
that takes place in the brain. By making it possible toexchange these prediction capabilities, we also aim to
expand individual-independent capabilities to make itpossible to exceed one’s limitations and those in
human capabilities in general.
3.3 The world we aim for and its application
Through expanded distance-independent capabilities and individual-independent capabilities, we aim

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

to achieve a world in which remote objects can be
operated freely by manipulating space-time in cyberspace to overcome the physical distance in the realworld. This could entail remotely operating a device
such as a robot arm in space or on the other side of theworld in real time with the sensations of grabbing,moving, or crafting with the human hand. We will
also work on extended capability technologies in
DTC by studying semi-autonomous control technologies required for the negative latency world beyond
zero through prediction considering the interactions
between people and objects in their surrounding
remote environments.
References
[1]
[2]
[3]
[4]
[5]

[6]

White paper on Digital Twin Computing, https://www.ntt.co.jp/
svlab/e/DTC/whitepaper.html
K. Ito, “Motor Control Networks,” Journal of the Society of Instrument and Control Engineers, Vol. 56, No. 3, pp. 161–162, 2017 (inJapanese).
H. Ikeda and J. Yoshio, “Synchronization in Multimedia,” The Journalof the Institute of Electrical Engineers of Japan, Vol. 126, No. 5, pp.
288–291, 2006 (in Japanese).
K. M. Stanney, R. R. Mourant, and R. S. Kennedy, “Human Factors
Issues in Virtual Environments: A Review of the Literature,” Presence,Vol. 7, No. 4, pp. 327–351, 1998.
H. Ujike, K. Ukai, and S. Saida, “Effects of Motion Types and Image
Contents on Visually-induced Motion Sickness,” Special Issue on
Safety, Health and Comfort of Cyberspace, The Journal of the Virtual
Reality Society of Japan, Vol. 9, No. 4, pp. 377–385, 2004.
S. Kasahara, J. Nishida, and P. Lopes, “Preemptive Action: Accelerating Human Reaction Using Electrical Muscle Stimulation without
Compromising Agency,” Proc. of the 2019 CHI Conference on
Human Factors in Computing Systems, pp. 1–15, Glasgow, Scotland,
UK, May 2019.

34

Feature Articles

Yukio Koike

Senior Research Engineer, Cybernetic Intelligence Research Project, NTT Service Evolution
Laboratories.
He received an M.S. from Nagoya University,
Aichi, in 2004 and joined NTT the same year. His
research interests include network technology forInternet-of-Things services and biological signal
processing.

Shigekuni Kondo

Senior Research Engineer, Planning Section,
NTT Service Evolution Laboratories.
He received a B.E. in applied physics and an
M.S. in integrated design engineering from Keio
University, Kanagawa in 2003 and 2005 andjoined NTT in 2005. His research interests
include software modularity, home networks,
and user interfaces. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE).

Tatsushi Matsubayashi

Distinguished Researcher, Cybernetic Intelligence Research Project, NTT Service Evolution
Laboratories.
He received a Ph.D. in science from Tokyo
Institute of Technology in 2006. He joined NTT
in 2005 and developed information-visualization, big-data-analysis, and optimization technologies. He is currently engaged in humanmachine interface and brain-machine interface
technology.

NTT Technical Review

Vol. 18 No. 9 Sept. 202035

Feature Articles: Digital Twin Computing for Advanced
Interaction between the Real World and Cyberspace

Social Issues with Digital Twin
Computing
Katsumi Takahashi
Abstract

This article introduces the social issues with Digital Twin Computing to bring about new digitalsocieties—a fusion of the real and virtual. Issues are presented as those of the three elements of data,
autonomous agents, and virtual societies from the perspectives of security/privacy and ethics/laws.
Essentially, social issues entail the human quest to freely engage in activities in both the real and virtualto achieve more comfortable and affluent lifestyles.
Keywords: security, privacy, ethics

1. What are the social issues with Digital
Twin Computing?
NTT laboratories are conducting research anddevelopment of Digital Twin Computing (DTC) toachieve new digital societies that fuse the real andvirtual. Social issues with DTC entail ensuring
human activities are not hindered in either the real or
virtual or discovering such hindrances in advance.
Humans now live in both real and virtual societies.
We, NTT laboratories, believe a stress-free combination of activities in both these societies will expand
human activities and bring about more comfortable
and affluent lifestyles. If there are inconsistencies
between the real and virtual, for example, activities inthe virtual that are not recognized in the real or
activities in the virtual that become uncontrollable
and destroy real activities, etc., such inconsistencies
could become sources of stress and anxiety for individuals and society. The essence of social issues with
DTC is how these inconsistencies (which we calldisconnected parallel worlds) can be foreseen. We
believe that the DTC vision we have put forth will
become socially stable by developing technologies toprevent such inconsistencies or limit the damage theycause.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

2. Components of DTC
To achieve new digital societies with DTC, we
adopted the concept of digital twins. A digital twin is“a mapping process that accurately represents incyberspace the shape, state, function, etc. of a realworld object (thing)” [1]. Virtual societies are createdby gathering digital twins. To address the social
issues with DTC, we perceive DTC as having threeelements (Fig. 1).
• Data• Autonomous agents consisting of data and interaction functions• Virtual societies consisting of collections ofdigital twins
Data express people and objects. In contrast to conventional digital twins that have been developed for
objects, DTC focuses on digital twins of humans. We
believe that digital twins must be used for general
purpose as well as express complex content because
we expect each digital twin to work in a broadersocial context rather than just as a solution to a problem. The goal of this idea is expressed as reproducing
human internalities in our DTC vision.
We presume that digital twins are autonomous
agents, in other words, systems that operate autonomously. This is because we assume that digital twins
will represent and strengthen human activities in virtual societies. We believe that digital twins will need36

Feature Articles

Data

Autonomous agents

Virtual societies

Data

Security and privacy

Ethics and laws

SecurePersonal dataprotection
Whose data?

Correct

Moderate

Fig. 2. Data of digital twins with DTC.

Fig. 1. Elements of DTC.

to be able to interact with each other (using functions
called operations) and, at least in part, will inevitably
need to be able to interact autonomously. Digital
twins will also carry out simulations and other calculations and actively feed the results back to the realtargets. Therefore, they will bridge the gap between
the real and virtual. Not only can digital twins beviewed as autonomous agents but also as part ofcyber-physical systems when viewed together withreal humans or objects that have received feedback.
We are also studying the generation of digital twins
with properties that do not exist (called derivative
digital twins) by other digital twins. This is a challenge.
A collection of digital twins and their activities iscalled a virtual society. DTC will provide environments where digital twins can be placed and operate.
Digital twins are autonomous agents that hold datafor people and objects internally, thus can form somekind of society. This will not just entail the aforementioned digital twin interactions. We believe that bygiving the operating environments of digital twins orderivative digital twins high level functionality oreven linking them with external applications, it will
be possible to create various virtual societies.
3. Considerations of social issues
As explained above, the goal of DTC is to create
new digital societies that combine the real and virtualand avoiding the creation of inconsistencies between
these two domains, in other words solving parallelworld problems, is a social issue.
What are the approaches for solving parallel-world
problems? There should be approaches for monitoring and controlling all digital twin activities in theenvironment provided by DTC. We do not deny thatthese approaches could be used in dedicated andclosed services that use digital twins, but we will nottake these approaches. Instead, we believe that each
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

element of DTC will follow conventional data processing and computing rules and that by harmonizing
the whole, we can avoid creating disconnected parallel worlds.
Accordingly, we will study each DTC element—
data, autonomous agents, and virtual societies—asissues to debate. We will attempt analysis from thetechnical (security/privacy) and rule (ethics/laws)
perspectives. Although, the technical perspective
focuses on information security [2] and privacy
frameworks [3], we will address broader concepts.
Obviously, the rule perspective must follow current
laws; however, we will also address ethics and laws
without making strong distinctions between them.
4. Data issues
The data for digital twins assumed with DTC will
have the following properties (Fig. 2).
• Data will express humans and objects• Personal data will be included• A wide scope will be described to address general problems• Details will be described to address complex
problems• Sensors will be used proactively• Data from a variety of sources will be integrated• Data may also be generated from other digitaltwins
A digital twin is a lump of data. Because we areproactively thinking about digital twins of humans
with DTC, a large amount of personal data will be
included. Moreover, because we want to achieve
more advanced computing with digital twins, we aim
to make digital twins be general-purpose as well as beable to express complex content. When expressing
human internalities, digital twins will consist of sensitive personal data that must be handled carefully.
We also take a proactive approach to generate data
in DTC. We are studying fusing and using sensor andcapture data, measurement data from wearable devices, and behavioral data from the Internet from the real37

Feature Articles

world. We are also studying not only the simple accumulation of data but also data generation (derivative
digital twins) by other digital twins.
From the above, we assume the following issues.
4.1 Security and privacy issues
As mentioned above, a digital twin is a lump of
data, thus requiring information security. Of course,
this will entail discussions about the basics of security—confidentiality (free of information leaks) andavailability (available when required). Therefore,
how will the integrity of a digital twin be perceived?
If we simply think of it as data integrity (accurate andwithout omissions), then solutions are easy, but digital twin integrity becomes an interesting issue when
thinking of it as the integrity of objects and humans
in digital space; a little beyond the scope of information security. Ultimately, the question of integrity is
whether all the data required to solve a problem is
described with a digital twin. Another idea recommended in the context of data privacy is called data
minimization, which is the pursuit of moderation in
the handling of data. The goal is to simultaneously
pursue moderation and precise collection of data.
4.2 Ethics and law issues
Since digital twin data are also personal data, it will
be necessary to comply with privacy and data protection regulations, and consistency will be required
even if these personal data are not subject to regulation. The main point of this issue is the nature of thedesign objectives, i.e., regarding how and to what
extent the person (subject) of the digital twin canunderstand the generation of the digital twin and its
purpose.
Such digital twin data can be generated from sensors installed by other people or data from other digital twins, which would make the digital twin a collaborative work so to speak. Thus, the issue here iswhose data are they?
5. Issues with autonomous agents
Digital twins assumed with DTC are autonomous
agents and will have the following properties (Fig. 3).
• Digital twins are systems that operate autonomously• They have functions called operations and interact with other digital twins• They can generate new digital twins (derivative
digital twins)• They comprise virtual societies
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Autonomous
agentsSecurity and privacy

Ethics and laws

Trustworthiness

Entrustment

Fig. 3. Autonomous agents with DTC.

• Unified with the real, they become part of cyberphysical systems
A digital twin is also a software system that
expresses a target human or object as data. This software has functions called operations and operates
autonomously by interacting with other twins. The
range of autonomy will gradually be implemented
from here on, although one currently planned capability is the generation of digital twins with properties
that do not exist (derivative digital twins). Digital
twins will form virtual societies. They will also feeddata back to real entities and become part of cyberphysical systems through unification with the real.
From the above, we assume the following issues.
5.1 Security and privacy issues
As with many other systems, we believe the expectation for autonomous agents will be their trustworthiness. Naturally, this means operating as expected(or enough to believe it so) at any time. Trustworthiness is a concept that includes a number of requirements that is not discussed in detail here, however, in
this case it is characterized by authentication andauthorization (information security), artificial intelligence (AI) malfunction, physical safety (cyber-physical systems), etc.
5.2 Ethics and law issues
The first issue is to entrust work to a digital twin.
The presence of objective judgments is a clue to this.
This is likely to involve transparency and explainability often discussed in the context of AI ethics. The
second issue is the social framework of entrustment.
This involves considering whether the autonomous
agent is a tool, representative of a human, the self ofa human, etc., and delving deeper into the relationship between the autonomous agent and the law. A
derivative argument of this issue is whether it is evenethical to use a digital twin. This problem is like thatof being allowed to use your smartphone to study athome but not in an examination room. The third issue
is of whether the work is one that a twin must not be
38

Feature Articles

made to do, aside from breaking the law. For example, in the case of derivative digital twins, if anotherperson creates a digital twin of someone else, howwill it be interpreted? Are there digital twins that areappropriate to create or those that should not be created (including the data-related issue of who owns thedata that are created)?

Virtualsocieties

Security and privacy

Ethics and laws

Trustworthiness

Rule design

Consistency

Fig. 4. Virtual societies with DTC.

6. Issues with virtual societies
Virtual societies conceived with DTC are collections of digital twins and their activities, as discussed
above. Virtual societies may be achieved throughlinkages with external applications or be thought ofas cyber-physical systems through linkages withactual humans. Although there are conceivably many
variations of virtual societies and many subjects
responsible for them, the following two generalissues (Fig. 4) are discussed in detail.
6.1 Security and privacy issues
With virtual societies, trustworthiness is typically
expected to be the same as with autonomous agents,
but cybersecurity measures are more important. A
major additional issue is that of ensuring consistency.
Consistency means that there are no contradictions
between the state of a digital twin and that of its subject. Since there can be more than one digital twin,
this also includes consistency between digital twins.
For example, if twin A sells a belonging of a subject,
and at the same time twin B has it destroyed, consistency will cease to exist (transaction management).
The first question in a virtual society is to what extent
transaction management is required and is key tosolving parallel-world problems.

society. This is the problem of how to constrain thefree behavior of agents. Examples of this could
include distribution of limited resources (ensuring
fairness) or the necessity of basic prohibitions (e.g.,
the elimination of discrimination). There is also theissue of whether to allow rules to override individual
protections when it is in the public interest, for example, to respond to disasters or pandemics.
7. Future developments
This article described the aims of DTC and discussed the social issues that should be addressed toachieve those aims as those of the three elements ofdata, autonomous agents, and virtual societies, anddescribed them from the perspectives of security/privacy and ethics/laws. By continuing to think about
these issues, we believe new digital societies thatcombine the real with the virtual are achievable.
References
[1]

[2]

6.2 Ethics and law issues
An important issue with a virtual society consistingof autonomous agents is the design of the rules of the[3]

T. Nakamura, “Digital Twin Computing Initiative,” NTT Technical
Review, Vol. 18, No. 9, pp. 13–18, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202009fa1.html
ISO/IEC 27000:2018: “Information Technology - Security Techniques - Information Security Management Systems - Overview and
Vocabulary,” 2018.
ISO/IEC 29100:2011: “Information Technology - Security Techniques - Privacy Framework,” 2011.

Katsumi Takahashi

Executive Research Scientist, Senior Manager
of Planning Section, NTT Secure Platform Laboratories.
He received a B.S. in mathematics from Tokyo
Institute of Technology in 1988 and a Ph.D. in
information science and technology from the
University of Tokyo in 2006. He joined NTT in1988 and has studied information processing
technologies including security and privacy.

NTT Technical Review

Vol. 18 No. 9 Sept. 202039

Regular Articles

Spatio-temporal Equalization
Technology for High-capacity
Underwater Acoustic Communication
Hiroyuki Fukumoto, Yosuke Fujino,
Kazumitsu Sakamoto, Marina Nakano,and Toshimitsu Tsubaki
Abstract

We are currently researching and developing Mbit/s-class underwater acoustic communication
technology enabling high-deﬁnition video and other large data transmission. This article proposes aspatio-temporal equalization technology, which is key for achieving high-speed underwater transmission.
Keywords: underwater acoustic communication, spatio-temporal equalization, distortion compensationin double selective channel

1. Introduction
Acoustic waves have long been used by humans as
a transmission medium underwater, similar to howdolphins use them. With the advantages of long communication range and not being affected by water
turbidity and ambient light, acoustic communication
has been applied for underwater image transmission
and communication between deep-sea exploration
vessels and offshore support vessels [1]. However,the communication rate with current commercial
modems is only tens of kbit/s (bits per second);
hence, a large amount of data, such as high-deﬁnition
video, cannot be transmitted in real time [2]. Against
this, we are currently researching and developing
high-capacity Mbit/s-class underwater acoustic communication technologies by utilizing our expertise inwave-propagation and signal-processing technologies that have been applied to terrestrial radio wireless communication. Our technologies make it possible to transmit high-deﬁnition video and other large
data, allowing full wireless remote control of undersea equipment such as remotely operated vehicles(ROVs), as shown in Fig. 1.
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

As a key technology to accelerate the transmissionrate, this article proposes a spatio-temporal equalization technology that can provide stable distortioncompensation regardless of transceiver movement or
degree of double selectivity in broadband acoustic
communication.
2. Underwater channel characteristics andissues regarding high-speed transmission
The underwater acoustic channel has double selectivity, in other words, the channel impulse response
varies with time. The cause is that the multipath
waves generated by reﬂection from the sea surface,sea bottom, and underwater structures arrive at thereceiver accompanied with Doppler shift due to the
movement of the transceiver and ﬂuctuation of sea
surfaces, as shown in Fig. 2 [3]. Since the propagation speed of sound waves in the sea (about 1500 m/sec) is about 200,000 times slower than that of radio
waves (about 3 × 108 m/sec), both degree of delay
spread and that of Doppler spread of underwater
channels, which are key parameters representing
double selectivity, are much larger than those of land40

Regular Articles

Remote monitoring, controlling
Marine buoy for relay

Video transmission, control

Underwater
construction
Fishery(aquaculture monitoring, etc.)
Resource development(equipment inspection)

Fig. 1. Our vision.

Movement
Reflection from sea surface
ReceiverDelay spread

Power
Movement

Transmitter

Direct path

Reflection path

Direct path
Delay timePower

Doppler spread

Reflection from sea bottom

ROV etc.

Direct path
Reflection path

Dopplerfrequency

Fig. 2. Acoustic propagation through underwater channels.

mobile radio channels. Thus, adaptive signal processing is essential for tracking time variations.
To accelerate transmission speed, we are now considering broadband acoustic transmission using ahigh carrier frequency of 100 kHz or more, which israrely used in current commercial systems. Since
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Doppler spread increases in proportion to the carrier
frequency from the viewpoint of sound propagation
theory, broadband transmission, which must use ahigh carrier frequency, makes time selectivity stronger than that with conventional narrowband transmission systems. Thus, the state of a channel response41Regular Articles• Conventional low-speed transmission: Delay spread <<1/(Doppler spread)
Delay spread

Period when the channel is almost constant (Training

1/Doppler spread)

PayloadAdaptive updating

Enabling compensation based on channel estimation• High-speed transmission (our target): Delay spread >1/(Doppler spread)Using a wider frequency band than conventional underwater communication
Prop. speed/(frequency × mobile speed)
Use of high carrier frequency

Delay spread
Period when the channel is almost constant( 1/Doppler spread)Training

PayloadCompensation is difficult because the stateof a channel changes during estimation.

Fig. 3. Issues regarding high-speed transmission.

rapidly and drastically varies even during a training
period. Conventional adaptive equalization technologies, such as decision-feedback equalization [4], forestimating the inverse characteristics of the underwater channel cannot keep up with channel variation if
the delay spread is less than the reciprocal of Doppler
spread, as shown in Fig. 3. Therefore, there is a needfor technology that can provide stable compensation
regardless of transceiver movement, or degree of
double selectivity, to accelerate the data transmission
rate.
3. Our spatio-temporal equalization technology
The essential diffculty with distortion compensation is estimating and tracking overall channel
responses in the time or frequency domain. Thus,
equalization technologies that are based on priorknowledge of the channel do not work well. The key
with our spatio-temporal equalization technology is
to shorten delay spread and Doppler spread by eliminating some of the delayed waves in the spatial
domain before carrying out equalization in the time
domain. By doing so, both delay spread and Doppler
spread of the multipath channel can be made substantially smaller than those of actual channel response,
NTT Technical Review

Vol. 18 No. 9 Sept. 2020as shown in Fig. 4, and this makes it possible toequalize acoustic signals through multipath channels
regardless of double selectivity.
We now brieﬂy describe the proposed spatio-temporal equalization technology. Figure 5 shows a
block diagram of our technology. All the ﬁnite
impulse response (FIR) ﬁlter coeffcients cascaded toeach input channel are adaptively updated using
adaptive algorithms, such as the recursive leastsquares algorithm, to minimize the square error
between the desired and equalized signals. It is similar to the design of adaptive equalization [5], except
for the length of the feed forward FIR ﬁlter. From theoperational comparison on the left of Fig. 6, the proposed technology uses FIR ﬁlters, the ﬁlter length ofwhich is much shorter than the actual delay spread. In
this case, the signal components of the indirect path,the delays of which are longer than the length of a
FIR ﬁlter, do not correlate with the desired signal.
Hence, they are suppressed in the spatial domain by
FIR-ﬁlter-operation updating with an adaptive algorithm. On the other hand, the signal components, thearrival delay of which is less than the length of a FIR
ﬁlter, correlate with the desired signal, therefore they
are equalized in the time domain, as shown at the bottom right of Fig. 6. Long-delay waves are thus
42

Regular Articles

Proposed

Conventional

Cancellation in the spatial domain

Equalizationin the time domain
Doppler frequency

Ne
sp t Dor ea pp
ler
d

Do
pp
ler

sp
rea
d

Doppler frequency
Equalizationin the time domain

Delay spread

Net delay spread

Delay time

Delay time

: Multipath component
Net delay spread
Period when the channel is almost constant( 1/net Doppler spread)Training

PayloadAdaptive updating

Enabling compensation based on channel estimation

Fig. 4. Comparison with conventional equalization and proposed spatio-temporal equalization technologies.

FIR filter
Equalized signal
FIR filter
Adaptivealgorithm

Desired signal

Error signal

Fig. 5. Practical implementation of our spatio-temporal equalization technology.

cancelled in the spatial domain while equalizing
residual paths in the time domain. Such a path-selective receiving structure makes stable compensation ofunderwater acoustic channels possible.
4. Field experiments
To verify the effectiveness of the proposed spatiotemporal equalization technology, we conducted two
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

transmission experiments using offline signal processing on a measurement barge moored in Suruga
Bay off of Shizuoka Prefecture, Japan. We experimentally evaluated the bit error rate (BER) withrespect to the speed of the vertically moving transmitter and conﬁrmed that a stable BER can be achieved
regardless of this speed or strength of double selectivity. The experimental system is shown in Fig. 7,
experimental environment for both experiments
43

Regular Articles

FIR

Power
Direct path

FIR
Reflection path

Equalized
signalEqualizing(combined) path
Equalizer output of conventional technology

Cancellation path
Delay time
FIR

Equalizing(combined)
path

Length of FIR filter (conventional)

FIR

Equalized
signalLength of FIR filter (proposed)
Equalizer output of proposed technology

Fig. 6. Operation of our spatio-temporal equalization technology.

UnderwaterTransmitter

Hydrophone
Transducer

AWG

PA

Receiver
LPF
LNA

LPF
LNA

LPF

Long
preamble

Payload(20,000 symbol)

Postamble

2.56 ms

10.2 ms

100 ms

2.56 ms

AWG: arbitrary waveform generator
DSO: digital storage oscilloscope
LNA: low noise amplifier

LPF
LNA

Short
preamble

DSO

PC

LNA

Waveform
data

Waveform
data

LPF: low pass filter
PA: power amplifier
PC: personal computer

Fig. 7. Experimental system.

(experiments 1 and 2) is shown in Fig. 8, and experimental speciﬁcations are presented in Table 1. Waveform data were recorded while the transmitter wascontinuously moving in the vertical direction, and therecorded data were analyzed. The mooring rope wasmanually moved vertically so that various speed patterns would be generated.
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

The observed BERs are shown in Figs. 9 and 10 for
experiments 1 and 2, respectively. The horizontal axis
is the speed of the vertically moving transmitter estimated using the known signal of the frame, and thevertical axis is the BERs calculated from the frame
demodulation results. The BERs ﬂuctuated regardless of the speed. This is because the level ratio of the44

Regular Articles

Hydrophone (receiver)c

Sea surface

b

0.1 m
0.1 m

Receivera
32 m

Transducer (transmitter)

Experimental environment

Verticalmovement

TransmitterSea bottom

Experiment 1

a

b

c

Delay spread

27 m

2m

10 m

2.33 msec(466-symbol delay)

4m

15 m

4.65 msec(930-symbol delay)

Experiment 2

Fig. 8. Experimental environment.

Table 1. Experimental specifications.
Parameter

Value

Carrier frequency

300 kHz

Bandwidth

200 kHz

Modulation

Quadrature phase-shift keying

Transmission system

Single carrier transmission

Transmission rate

400 kbit/s (payload only)

Equalization technology

Spatio-temporal equalization

Number of taps

54 tap/ch

Transducer and hydrophone arrangement TX: 1, RX: 4 (2 x 2 rectangular array)direct wave and sea-surface reﬂected wave varied
greatly from frame to frame due to changes on the sea
surface. All the BERs were below the forward error
correction (FEC) limit of 7% [6]. Also, the correlation coeffcient between speed and BER was 0.21, andthere was almost no causal relationship between thedeterioration in BER with respect to speed or double
selectivity of the channel. Through these experiments, we confirmed the possibility of high-speed
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

broadband acoustic transmission under actual sea
conditions.
5. Conclusion
In this article, we presented our future vision for anunderwater wireless communication system. Channel
characteristics and issues with broadband acoustic
transmission were also presented. To obtain stable
45

Regular Articles

1.E+00

1.E+00BER performance

1.E-01

7% FEC limit

1.E-02

7% FEC limit

1.E-02BER

BER

BER performance

1.E-01

1.E-03

1.E-03

1.E-04

1.E-04

1.E-05

1.E-05

Error free

Error free
0

0.5

1

0

1.5

Speed (m/s)

0.5

1Fig. 9. BERs observed in experiment 1.

Fig. 10. BERs observed in experiment 2.

compensation regardless of double selectivity, we
proposed a spatio-temporal equalization technology
and presented experiments conducted at sea and theresults, which indicate that stable communication canbe achieved regardless of speed. We also conﬁrmed
the possibility of broadband underwater communication.
We mainly considered a single-input multiple-output transmission system in this article. However,multiple-input multiple-output (MIMO) extension iseasy to implement thanks to the fact that our equalization technology eliminates uncorrelated interference
signals simultaneously. With this property, we successfully transmitted 5.12 Mbit/s signals over 18 m
using 8 × 16 MIMO transmission with offline signal
processing, demonstrating the achievability of Mbit/sclass transmission. See our paper to be published [7]for a detailed description of a field experiment and amethod of extending MIMO transmission.

References

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

1.5

Speed (m/s)

[1]
[2]
[3]

[4]
[5]

[6]

[7]

M. Suzuki, T. Sasaki, and T. Tsuchiya, “Digital Acoustic Image
Transmission System for Deep-sea Research Submersible,” Proc. of
the OCEANS 92, pp. 567–570, Newport, RI, USA, Oct. 1992.
S. Sendra, J. Lloret, J. M. Jimenez, and L. Parra, “Underwater
Acoustic Modems,” IEEE Sensors J., Vol. 16, No. 11, pp. 4063–4071,
June 2016.
M. Stojanovic and J. Preisig, “Underwater Acoustic Waveform Data
Communication Channels: Propagation Models and Statistical
Characterization,” IEEE Commun. Mag., Vol. 47, pp. 84–89, Jan.
2009.
M. Stojanovic, J. A. Catipovic, and J. G. Proakis, “Phase-coherent
Digital Communications for Underwater Acoustic Channels,” IEEE J.
Ocean. Eng., Vol. 19, No. 1, pp. 100–111, Jan. 1994.
M. Johnson, L. Freitag, and M. Stojanovic, “Improved Doppler
Tracking and Correction for Underwater Acoustic Communications,”
Proc. of 1997 IEEE International Conference on Acoustics, Speech,
and Signal Processing, pp. 575–578, Munich, Germany, Apr. 1997.
A. Leven, F. Vacondio, L. Schmalen, S. ten Brink, and W. Idler,“Estimation of Soft FEC Performance in Optical Transmission
Experiments,” IEEE Photon. Technol. Lett., Vol. 23, No. 20, pp.
1547–1549, Oct. 2011.
H. Fukumoto, Y. Fujino, M. Nakano, K. Sakamoto, and T. Tsubaki,“Field Experiment Demonstrating Mbps-class Underwater Acoustic
Communication with Spatio-temporal Equalization,” to appear inproc. of OCEANS 2020, Oct. 2020.

46

Regular Articles

Hiroyuki FukumotoResearcher, Wave Propagation Laboratory,
NTT Network Innovation Laboratories.
He received a B.E. in engineering from Kobe
University, Hyogo, in 2013, and an M.E. in informatics from Kyoto University in 2015. He has
been with NTT Network Innovation Laboratories
since 2015. His research interest includes adaptive signal processing, information theory, andspace division multiplexing technologies. His
current research is on architecture design andimplementation for high-speed underwater
acoustic communication modems. He is a member of the Institute of Electronics, Information
and Communication Engineers (IEICE).

Yosuke Fujino

Senior Research Engineer, Wave Propagation
Laboratory, NTT Network Innovation Laboratories.
He received a B.E. and M.E. in electrical andelectronic engineering from Shizuoka University
in 2002 and 2004. He joined NTT Network Innovation Laboratories in 2004 and engaged inresearch on physical layer signal design andtransceiver architecture for Internet of Things(IoT) wireless systems. The current focus of his
research is wireless communication technology
for frontier areas such as underwater, underground, and space.

Marina Nakano

Researcher, Wave Propagation Laboratory,
NTT Network Innovation Laboratories.
She received a B.E. and M.E. from Miyazaki
University in 2015 and 2017. She joined NTT
Network Innovation Laboratories in 2017 and iscurrently engaged in research and developmentof IoT/M2M wireless systems. She is a memberof IEICE.

Toshimitsu Tsubaki

Senior Research Engineer, Wave Propagation
Laboratory, NTT Network Innovation Laboratories.
He received a B.S. and M.S. in geophysics
from Tohoku University, Miyagi, in 1994 and1996. He joined NTT in 1996 and has been
engaged in research and development of media
access control for broadband wireless accesssystems, radio frequency identiﬁcation tagging
systems, non-contact high-speed transmission
systems, and network assistance technologies for
IoT services. His current research interest is inunderwater high-speed ultrasonic communication systems. He is a member of IEICE.

Kazumitsu Sakamoto

Research Engineer, Wave Propagation Laboratory, NTT Network Innovation Laboratories.
He received a B.E. and M.E. in electrical engineering from Kyushu University, Fukuoka, in2008 and 2010. In 2010, he joined NTT Network
Innovation Laboratories and has been engaged inresearch on MIMO communication systems,
millimeter-wave high-speed wireless systems,
and IoT and machine-to-machine (M2M) wireless access systems. He received the Young
Engineers Award and the Communications Society Best Letter Award from IEICE in 2015. He isa member of IEICE.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

47

Global Standardization Activities

Meeting Report on ITU-T
FG-AI4EE—ICT Applications Using
Artificial Intelligence in Climate
Change Countermeasures and Their
Standardization Trends
Xiaoxi Zhang, Kazuhiro Takaya, Yoshihiro Kondo,
and Yoshinori Sugimoto
Abstract

Although emerging technologies, such as artificial intelligence (AI) and blockchain, are expected to
improve service availability and work efficiency in various fields, they consume a large amount ofenergy during data collection from many sensors, learning, and data analysis. Therefore, it is necessary
to evaluate the environmental impact as well as other effects of introducing such technologies. A shortterm focus group in the International Telecommunication Union - Telecommunication Standardization
Sector (ITU-T), Focus Group on Environmental Efficiency for Artificial Intelligence and other Emerging
Technologies (FG-AI4EE), was established in May 2019 to conduct a preliminary study of the environmental efficiency of AI and other emerging technologies and held its first meeting in December 2019.
This article describes not only what FG-AI4EE is and what was discussed at its first meeting but also
how NTT has been working on environmental issues based on the Innovative Optical and Wireless Network (IOWN), including the response to crises such as the current COVID-19 pandemic.
Keywords: environmental efficiency, AI and emerging technologies, focus group, ITU-T SG5

1. Environmental impact of artificial intelligenceand emerging technologies
Society 5.0 [1] proposed by the Japanese Cabinet
Office aims to achieve economic growth and solve
social issues in parallel by using a system that closely
integrates cyberspace and physical space. Information distribution using artificial intelligence (AI),Internet of Things (IoT), big data, etc., is intended toenable a high quality of life. Natural disasters due toclimate change have also been causing substantial
damage worldwide, so it is necessary to develop
effective methods for environmental protection using
information and communications technology (ICT).
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

In particular, AI and other emerging technologies,
such as blockchain, edge/cloud computing, and fifthgeneration mobile communication systems (5G), areexpected to help solve such environmental problems
by enabling near-future environmental prediction andvisualization of meteorological phenomena. However, the energy consumption of AI and other emerging
technologies is also expected to have an environmental impact due to the increase in the size of applications and amount of calculation, which is similar tothat of current services and applications. Furthermore, AI consumes a large amount of energy not only
when analyzing a large amount of data collected fromsensors and system logs but also during pre-learning48

Global Standardization Activities

and tuning. Therefore, to minimize the environmental
impact of using ICT, a method is required for evaluating and measuring the environmental efficiencies
with respect to both ICT performance and environmental impact, which have not been defined by existing international standards and/or guidelines in theenvironmental field.
2. Purpose of establishing FG-AI4EE
Within the International Telecommunication Union(ITU) under the United Nations (UN), telecommunications carriers and ICT vendors are developing
ITU-T Recommendations for guidance to solve various environmental issues regarding ICT in Study
Group 5 (SG5) of the Telecommunication Standardization Sector (ITU-T). The methodology for assessing the environmental impact of the ICT sector has
been completed as Recommendation ITU-T L.1450.
In addition, Recommendation ITU-T L.1451, which
was developed in 2019 with contributions from NTT,
defines the method of assessing the aggregated positive environmental impact of ICT in other sectors.
Since the quantitative evaluation of ICT’s potential
contribution to environmental protection has progressed, a short-term Focus Group on Environmental
Efficiency for AI and other Emerging Technologies(FG-AI4EE) was established in May 2019 to conduct
a preliminary study called “Environmental Efficiency
for AI and other Emerging Technologies.” FG-AI4EE
started its activities to promote a global dialogue on
the environmental efficiency of AI and other emerging technologies.
Emerging technologies are expected to improve
work efficiency and expand potential applications invarious fields but consume a large amount of energy
during installation and operation and increase environmental impact. To achieve the UN Sustainable
Development Goals (SDGs), methods and global
evaluation standards are needed to evaluate and measure the environmental impact of solutions that use
emerging technologies. FG-AI4EE is expected tofunction as a platform for clarifying the standardization requirements for achieving the SDGs.
3. Overview of the first FG-AI4EE meeting
The first FG-AI4EE meeting was held in Vienna(Austria) from December 11 to 13, 2019. A workshopof the AI4EE Forum and a meeting of the U4SSC(United for Smart Sustainable City) index thematicgroup were held in conjunction with the FG-AI4EE
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

meeting. Unlike the ITU-T SG5 meeting, the FGAI4EE meeting involved not only telecommunications-related parties but also other entities: international organizations such as UN and governmentaffiliated organizations, and academic institutions
such as universities and think tanks. About 30 participants attended the meetings in person, including
two of the authors (Mr. Kondo and Mr. Sugimoto),
and about 15 attended remotely from around theworld.
There were two sessions in the AI4EE Forum. The
first session was held to explore the possible environmental impact of AI, and the second was devoted to
discussing use cases for assessing such impact. During the forum, the necessity of AI and the goal of
FG-AI4EE were shared among all participants. At the
FG-AI4EE meeting, there were discussions regarding
1) management team, 2) working group (WG) structure, 3) terms of reference (ToR), and 4) FG deliverables to be developed.
3.1 Management team
The FG chairpersons are Dr. Paolo Gemma (Huawei), who also chairs Working Party 2 of ITU-T SG5,
and Dr. Neil Sahota (IBM & University of California). Professors and experts in the fields of finance/
economics and environment/energy are among thosenamed as vice-chairpersons. Table 1 shows the chairpersons and vice-chairpersons agreed at this meeting.
3.2 WG structure
It was agreed that the three newly established WGs
shown in Table 2 would carry out detailed studies.
(1)	WG1: Requirements of AI and other Emerging
Technologies to Ensure Environmental Efficiency
WG1 studies the requirements for improving environmental efficiency when introducing AI and otheremerging technologies. In various use cases, introducing an emerging technology that fulfills theserequirements would be expected to result in reduced
energy consumption, which would reduce environmental impact. For example, key performance indicators (KPIs) such as bandwidth usage are collectedfrom an optical line termination and optical network
termination in a passive optical network (PON) system, then root-cause analysis using AI is carried out.
This analysis would identify minimum required
resources that satisfy the service-level agreement,
which would in turn reduce the power consumptionof the PON system and the number of on-site calls.
(2)	WG2: Assessment and Measurement of the49

Global Standardization Activities

Table 1. FG-AI4EE management team.
Co-Chairpersons• Paolo Gemma (Huawei, China)• Neil Sahota (IBM & University of California)
Vice-Chairpersons• Barbara Kolm (Austrian Economics Center & Austrian National Bank)• Kari Eik (Organization for International Economic Relations (OiER))• Joel Alexander Mills (AugmentCity AS)• Mats Pellbäck Scharp (Ericsson)• Alice Charles (World Economic Forum (WEF))• Claudio Bianco (Telecom Italia)• Peter Ulanga (Universal Communications Service Access Fund, United Republic of Tanzania)• Stefano Nativi (European Commission - Joint Research Centre)• Mandar Deshpande (Ministry of Communications, India)• UN-Environment Programme

Table 2. FG-AI4EE WG structure.
WG

Title

WG1

Requirements of AI and other Emerging Technologies to
Ensure Environmental Efficiency

WG2

Assessment and Measurement of the Environmental
Efficiency of AI and Emerging Technologies

WG3

Implementation Guidelines of AI and Emerging
Technologies for Environmental Efficiency

Environmental Efficiency of AI and Emerging
Technologies
WG2 studies environmental efficiency assessmentand measurement methods that help carriers andrelated stakeholders introduce AI and other emerging
technologies. It would be expected that the outcome
for achieving the SDGs can be attained by using theevaluation and measurement methods studied in
WG2 to improve the quality of these technologies
while considering their operational efficiency andenvironmental impact.
(3)	WG3: Implementation Guidelines of AI and
Emerging Technologies for Environmental Efficiency
WG3 studies implementation guidelines to improvethe environmental efficiency of AI and other emerging technologies. Regarding blockchain, which supports crypto-currencies, the guidelines on how to
improve environmental efficiency and the principleof environmental efficiency by the transition from“proof of work” (allowing more computationallyintensive users to work more favorably) to “proof of
NTT Technical Review

Vol. 18 No. 9 Sept. 2020

stake” (not wastefully consuming energy even whencompeting with others) will be provided.
3.3 ToR
FG-AI4EE will develop technical reports and technical specifications to address the environmental
efficiency as well as water and energy consumptionof emerging technologies to meet the SDGs. Specifically, the key objectives of the FG include the following.
• Identify and analyze the environmental impact of
deploying and implementing AI and otheremerging technologies• Provide a platform for facilitating global dialogue and raising awareness on the environmental impact of AI and other emerging technologies• Assist relevant stakeholders in minimizing the
global environmental impact of operating AI andother emerging technologies• D evelop a framework and standardization
approach to adopting AI and other emerging
technologies in an environmentally sound manner
50

Global Standardization Activities

Table 3. Deliverables to be developed.
WG

Title

SourceWG2

Technical Report on Energy Efficiency Assessment and Metrics

University of Thessaly

WG3

Technical Report on Visions of Best Practices on Artificial Intelligence andBlockchain in 2025

WG2

Technical Report on Methodology for Supporting the Implementation of Artificial
Intelligence and Blockchain Solutions at the Government Level

WG1

Guidelines Assessing Eco-Friendliness of Requirements

WG2

List of KPIs/Metrics

WG3

Eco-friendly Build/Implement Guidelines

WG1

Study on How AI Is Essential in Currently Deployed Network and Future
Evolutions to Optimize Resource Utilization and to Reduce Call Out and Related
Energy Dissipation

WG2

Technical Report on “Guidelines on the Environmental Efficiency of 5G Usage in
Smart Water Management”

Computer Processing, Data management and Energy Perspective

Austrian Economics Centre
European Commission - Joint Research Centre
University of California

WG1 Requirements of AI and Other Emerging Technologies to Ensure Environmental/WG2 Efficiency: Simulation Conference: Case Study of Environmental Impact of AI or/WG3 Blockchain, Which Will Include the Visualization of the KPIs
WG2

Establishment of 5G Energy Efficiency Assessment Model Based on AI and Big
Data Analysis

WG3

Application of AI Technology in Improving Energy Efficiency of IDC Infrastructure

Huawei

AugmentCity

China Telecom

IDC: Internet datacenter• Describe strategic directions for future standardization efforts in tackling the environmental
impact of these technologies

based on the proposal in the relevant contribution.
The WG responsible for handling each deliverable
will be decided from future discussions in the FG.

3.4 Deliverables to be developed
At the first FG-AI4EE meeting, participants agreed
to develop 28 deliverables. Table 3 lists some of thesedeliverables. Governmental and academic organizations that emphasize effective measures against climate change proposed evaluation metrics for theenergy efficiency of AI and blockchain technologies.
As a telecommunications carrier, China Telecom proposed developing methodologies for measuring andevaluating the energy efficiency of 5G networks
using AI and big data analysis and writing up implementation guidelines.
The following deliverables were also proposed: 1)
implementation and design requirements of assessment methodologies for environmental impact byvisualizing measurement data of relevant KPIs using
digital twin technology, 2) reporting templates forenvironmental-friendliness assessment and measurements, and 3) implementation guidance of networkoperation management using AI and machine learning.
Since a WG that handles the deliverables shown in
Table 3 has not been agreed upon, the deliverables are

4. Changes in social environment after the first
FG meeting

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Social systems worldwide have had to adapt to theglobal spread of COVID-19. Changes include theincrease in communication traffic caused by the transition to telework and online education, promotion oftelemedicine, increase in online shopping and homedeliveries, and increase in food disposal due to theclosure of restaurants and schools. These changes
will result in not only benefits but also considerable
disadvantages in terms of environmental impact. To
create a sustainable smart city, sustainable economic
activities must be maintained while reducing greenhouse gas emissions from using ICT.
The NTT Group has proposed the Innovative Optical and Wireless Network (IOWN) toward 2030 withmultiple pillars of technological developments, oneof which is “a dramatic improvement in energy efficiency in ICT infrastructure” [2, 3]. IOWN must be
achieved so that the environmental impact of ICT canbe minimized. IOWN should enable ICT to be used to
avoid or minimize any damage caused by future
51

Global Standardization Activities

global crises as well as the current COVID-19 pandemic.
For the sustainable development of enterprises
worldwide, the environmental impact of ICT services
will need to be evaluated accurately and appropriately and various social issues will need to be solved
by using IOWN from the perspective of environment,
society, and governance (ESG) management considering the environment and society.
5. Summary and future activities
Although emerging technologies such as AI are
expected to improve work efficiency and expand
business possibilities in various fields, they consume
a large amount of energy during installation andoperation, which in turn leads to increased environmental impact. To achieve the UN SDGs, it is essential to develop methodologies for evaluating andmeasuring environmental impact and international
evaluation standards. Most proposals at the first FGAI4EE meeting were for energy efficiency and climate change, and few proposals were for a circular
economy including measures for e-waste (electrical

Xiaoxi Zhang

and electronic waste). Since ICT plays an importantrole in contributing to the SDGs, FG’s activity shouldmake enterprises and organizations aware of ESG
management, as well as share best practices worldwide and develop international guidelines. Whilecontinuing to investigate the activities of this FG sothat IOWN can simultaneously help solve environmental problems and various social issues, NTT laboratories will continue to contribute to the international telecommunications community by sharing theresults of its ICT and environmental research anddevelopments through this FG activity.
References
[1]
[2]

[3]

Cabinet Office, Government of Japan, “Society 5.0,”
https://www8.cao.go.jp/cstp/english/society5_0/index.html
S. Iwashina, Y. Aragane, K. Minamihata, K. Shindo, and M. Fujiwara,“Initiatives to Achieve the IOWN (Innovative Optical and Wireless
Network) Concept,” NTT Technical Review, Vol. 18, No. 2, 2020.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202002iw1.html
Press release issued by NTT, “Technology Development Roadmap for
Realizing the IOWN Concept,” Apr. 16, 2020.
https://www.ntt.co.jp/news2020/2004e/200416a.html

Yoshihiro Kondo

Senior Research Engineer, ESG Management
Science Group, Resilient Environmental Adaptation Research Project, NTT Space Environment
and Energy Laboratories.
She received a B.S. and M.S. in social engineering from Tokyo Institute of Technology in2007 and 2009. Since joining NTT in 2009, she
has been engaged in research on risk-based management of telecommunication infrastructures.
She is currently researching methods of assessing the environmental and economic impact of
ICT use. She won the Distinguished Service
Award from the Telecommunication Technology
Committee in 2020. She is a member of the Institute of Electronics, Information and Communication Engineers (IEICE).

Senior Expert Engineer, IOWN Business
Development Business Unit, Network Innovation Business Headquarters, NTT Advanced
Technology Corporation.
He received an M.S. and Ph.D. in electricalengineering from Northwestern University inEvanston, Illinoi, USA, in 1990 and 1994. Afterhe worked for OKI Electric Industry Co., Ltd.,
where he developed optical access network systems, he joined NTT Advanced Technology in2006. Since then, he has been working on standardization of multiple areas such as environmental and energy-efficiency-related issues,
access network systems, and home network systems. He is a member of the Institute of Electricaland Electronics Engineers (IEEE).

Kazuhiro Takaya

Yoshinori Sugimoto

Senior Research Engineer, Supervisor, Zero
Environmental Impact Research Project, NTT
Space Environment and Energy Laboratories.
He received a B.E. and M.E. in electrical andelectronic engineering from Okayama University
in 1993 and 1995 and received a Ph.D. in electrical engineering from Kyoto University in 2019.
He joined NTT Telecommunication Networks
Laboratories in 1995. Since then, he has been
engaged in the study of electromagnetic compatibility of communication systems. He received
the Communication Society Activity Achievement Award in 2015 for his contributions as asecretary of IEICE Technical Committee on
Electromagnetic Compatibility (EMCJ). He
received the Encouragement Award in 2017 fromthe ITU Association of Japan for his contributions relating to ITU-T SG5. He is a Vice-chairman of ITU-T SG5. He is a member of IEICE.

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

Engineer, IOWN Business Development Business Unit, Network Innovation Business Headquarters, NTT Advanced Technology Corporation.
He received a B.S. and M.S. in information andcommunication engineering from the University
of Electro-Communications, Tokyo, in 2003 and2005. Since joining NTT Advanced Technology
in 2005, he has been engaged in research on highspeed data transmission in the Generalized Multiprotocol Label Switching network, and Service
OAM (Operations, Administration, and Maintenance) in Metro Ethernet Forum, etc. He is currently researching the role of ICTs in tacklingclimate change. He is participating in ITU-T
FG-AI4EE.

52

External Awards
DCASE 2019 Best Paper Award
Winners: Shota Ikawa, The University of Tokyo; Kunio Kashino,
NTT Communication Science Laboratories
Date: October 25, 2019
Organization: The Workshop on Detection and Classification of
Acoustic Scenes and Events (DCASE) Community
For “Neural Audio Captioning Based on Conditional Sequence-toSequence Model.”Published as: S. Ikawa and K. Kashino, “Neural Audio Captioning
Based on Conditional Sequence-to-Sequence Model,” Proc. of
DCASE2019, pp. 99–103, NY, USA, Oct. 2019.
CHI 2020 Honorable Mention Award
Winner: Kentaro Yasu, NTT Communication Science Laboratories
Date: April 16, 2020
Organization: Association for Computing Machinery’s Special
Interest Group on Computer Human Interaction (ACM SIGCHI)
For “MagneLayer: Force Field Fabrication by Layered Magnetic
Sheets.”Published as: K. Yasu, “MagneLayer: Force Field Fabrication by
Layered Magnetic Sheets,” Proc. of the 2020 CHI Conference on
Human Factors in Computing Systems, Apr. 2020.
IEEE SPS Young Author Best Paper Award
Winners: Daichi Kitamura, National Institute of Technology, Kagawa College; Nobutaka Ono, Tokyo Metropolitan University; Hiroshi
Sawada, Hirokazu Kameoka, NTT Communication Science Laboratories; Hiroshi Saruwatari, The University of Tokyo
Date: May 5, 2020
Organization: IEEE Signal Processing Society
For “Determined Blind Source Separation Unifying Independent
Vector Analysis and Nonnegative Matrix Factorization.”
Published as: D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and H.
Saruwatari, “Determined Blind Source Separation Unifying Independent Vector Analysis and Nonnegative Matrix Factorization,” IEEE/
ACM Trans. Audio, Speech, Language Process., Vol. 24, No. 9, 2016.
AI2000 Most Influential Scholar Honorable Mention
Winner: Tomohiro Nakatani, NTT Communication Science Laboratories
Date: May 10, 2020
Organization: AMiner.org, Tsinghua University

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

For his outstanding contributions to the field of speech recognition
between 2009 and 2019.
OSA Senior Member
Winner: Salah Ibrahim, NTT Device Technology Laboratories
Date: June 1, 2020
Organization: The Optical Society (OSA)
For his professional accomplishments with emphasis on opticallyswitched networks and their enabling technologies.
DICOMO2020 Best Presentation Award
Winner: Keisuke Tsunoda, NTT Service Evolution Laboratories/
NTT Smart Data Science Center
Date: June 26, 2020
Organization: Information Processing Society of Japan (IPSJ)
For “Estimation Number and Dwell Time of Visitors from CO2
Concentration Based on Partial Modeling with Variable Time Window.”
Published as: K. Tsunoda, N. Arai, and K. Obana, “Estimation Number and Dwell Time of Visitors from CO2 Concentration Based on
Partial Modeling with Variable Time Window,” Proc. of the Multimedia, Distributed, Cooperative, and Mobile Symposium (DICOMO)
2020, pp. 259–266, June 2020.
Civil Engineering Informatics (CEI) Paper Award
Winners: Daisuke Uchibori, NTT Access Network Service Systems
Laboratories; Sho Ashikaga, NTT EAST; Taishi Deguchi, Kazuhiro
Nishimoto, NTT InfraNet Corporation; Hiroyuki Takahashi,
Research and Development Planning Department, NTT; Masafumi
Nakagawa, Kazuhiko Goto, NTT EAST; Shuichi Yanagi, NTT
Device Technology Laboratories
Date: July 3, 2020
Organization: Japan Society of Civil Engineers
For “Development of a Manhole Cover with Pattern for Improved
Durability and Visibility of Worn Condition.”
Published as: D. Uchibori, S. Ashikaga, T. Deguchi, K. Nishimoto,H. Takahashi, M. Nakagawa, K. Goto, and S. Yanagi, “Developmentof a Manhole Cover with Pattern for Improved Durability and Visibility of Worn Condition,” Journal of Japan Society of Civil Engineers
F3 (Civil Engineering Informatics), Vol. 75, No. 1, pp. 1–12, 2019.

53

Papers Published in Technical Journals andConference Proceedings
Classically Simulating Quantum Circuits with Local Depolarizing NoiseY. Takahashi, Y. Takeuchi, and S. TaniarXiv:2001.08373v1 [quant-ph], January 2020.
We studied the effect of noise on the classical simulatability ofquantum circuits defined by computationally tractable (CT) statesand efficiently computable sparse (ECS) operations. Examples ofsuch circuits, which we call CT-ECS circuits, are instantaneous quantum polynomial-time (IQP), Clifford Magic, and conjugated Clifford
circuits. This means that there exist various CT-ECS circuits such that
their output probability distributions are anti-concentrated and notclassically simulatable in a noise-free setting (under plausible
assumptions). First, we consider a noise model where a depolarizing
channel with an arbitrarily small constant rate is applied to each qubit
at the end of computation. We show that, under this noise model, if
an approximate value of the noise rate is known, any CT-ECS circuitwith an anti-concentrated output probability distribution is classically simulatable. This indicates that the presence of small noise drastically affects the classical simulatability of CT-ECS circuits. Then, we
consider an extension of the noise model where the noise rate canvary with each qubit and provide a similar sufficient condition forclassically simulating CT-ECS circuits with anti-concentrated output
probability distributions.

Si-nanowire-FET Sensor Detecting High-frequency Oscillation of a Multilayer-graphene MEMS by Means of Reflectometry Technique
K. Nishiguchi and A. Fujiwara
Proc. of the 67th Japan Society of Applied Physics Spring Meeting, 12p-D311-7, March 2020.
Using a field-effect transistor (FET)-based sensor composed ofmultiple Si nanowire channels, we demonstrated the detection of
mechanical oscillations of a multilayer-graphene microelectromechanical system (MEMS). The multilayer graphene sheet is suspended above the FET’s channels, which is driven by a high-frequency signal via microwave probes connected to double LC matchingcircuits. Such a reflectometry technique enables detection of graphene’s mechanical oscillations at 240 MHz.

Control of Thermal Noise Originating from Single-electron
Brownian Motion
K. Nishiguchi, K. Chida, and A. Fujiwara
Proc. of the 6th International Symposium toward the Future ofAdvanced Researches in Shizuoka University (ISFAR-SU 2020), pp.
24–25, March 2020.
The performance of a data-processing circuit composed of Si fieldeffect transistors (FETs) has been improved by the miniaturization of
FETs. However, this FET miniaturization generates more noise,
which degrades FET performance. Among various kinds of noises,
thermal noise, which originates from the Brownian motion of electrons in electronic devices, is one of fundamental and unavoidable
noises. This noise is also a main factor in providing the lowest limitof energy for one bit of information in data processing, which is
known as Landauer’s limit. Therefore, thermal noise is one of the

NTT Technical Review

Vol. 18 No. 9 Sept. 2020

most important topics that could provide insight into achieving lowpower-consumption electric devices.
We introduce the microscopic analysis of thermal noise, i.e.,counting statistics of single-electron Brownian motion, using Si
FETs. Based on the analogy of Maxwell’s demon controlling singleelectron Brownian motion, we also discuss electric power generationfrom the Brownian motion.

Single-electron-resolution Noise Analysis and Application
Using High-sensitivity Charge Sensor
K. Nishiguchi, K. Chida, and A. Fujiwara
AAPPS (Association of Asia Pacific Physical Societies) Bulletin,Vol. 29, No. 3, pp. 4–9, June 2020.
We introduce an analysis of thermal noise using a high-sensitivity
charge sensor. Since the sensor is based on a Si field-effect transistor
with a channel size of approximately 10 nm, it exhibits high sensitivity to detect single-electron motion even at room temperature. By
connecting this sensor to a small capacitor composed of dynamic
random-access memory, thermal noise in the capacitor can be monitored in real time with single-electron resolution. Such real-timemonitoring reveals that when the capacitor is small enough that the
charging energy for storing one electron in the capacitor is greaterthan the thermal energy, thermal noise is suppressed and enhanced.
This represents a deviation from the law of energy equipartition. In
addition to this noise analysis, we conducted an experiment on power
generation using an analogy of Maxwell’s demon that detects andmanipulates single-electron motion, which should accelerate
research in the field of thermodynamics. These experimental results
indicate that the high-sensitivity charge sensor can function as asuperior platform for microscopic analysis of noise, small electronic
devices, and thermodynamics as well as a demonstration of theoretical expectation in basic research.

1-day, 2 Countries—A Study on Consumer IoT Device Vulnerability Disclosure and Patch Release in Japan and theUnited States
A. Nakajima, T. Watanabe, E. Shioji, M. Akiyama, and M. Woo
IEICE Transactions on Information and Systems, Vol. E103-D,
No. 7, pp. 1524–1540, July 2020.
With our ever-increasing dependence on computers, many governments around the world have started to investigate strengthening theregulations on vulnerabilities and their lifecycle management.
Although many previous works have studied this problem space formainstream software packages and web applications, relatively few
have studied this for consumer Internet of Things (IoT) devices. As
the first step towards filling this void, this paper presents a pilot study
on the vulnerability disclosures and patch releases of three prominent
consumer IoT vendors in Japan and three in the United States. Our
goals include (i) characterizing the trends and risks in the vulnerability lifecycle management of consumer IoT devices using accurate
long-term data and (ii) identifying problems, challenges, and potential approaches for future studies of this problem space. To this end,
we collected all published vulnerabilities and patches related to theconsumer IoT products of these vendors between 2006 and 2017 then54

Papers Published in Technical Journals and Conference Proceedings

analyzed this dataset from multiple perspectives, such as the severityof the included vulnerabilities and timing of the included patch
releases with respect to the corresponding disclosures and exploits.
We uncovered several important findings that may inform futurestudies. These findings include (i) a stark contrast between how thevulnerabilities in this dataset were disclosed in the two markets, (ii)three alarming practices by the included vendors that may significantly increase the risk of 1-day exploits for customers, and (iii)challenges in data collection including crawling automation andlong-term data availability. For each finding, we also provide discussions on its consequences and/or potential migrations or suggestions.

Experimental Demonstration of Secure Quantum Remote
Sensing
P. Yin, Y. Takeuchi, W. Zhang, Z. Yin, Y. Matsuzaki, X. Peng, X.
Xu, J. Xu, J. Tang, Z. Zhou, G. Chen, C. Li, and G. Guo
Physical Review Applied, Vol. 14, 014065, July 2020.
Quantum metrology aims to enhance the precision of various measurement tasks by taking advantage of quantum properties. In many
scenarios, precision is not the sole target; the acquired informationmust be protected once it is generated in the sensing process. Considering a remote sensing scenario where a local site performs cooperative sensing with a remote site to collect sensitive information at that
remote site, the loss of sensing data inevitably reveals sensitive
information . Quantum key distribution is known to be a reliable solution for secure data transmission; however, it fails if an eavesdropper
accesses the sensing data generated at a remote site. In this study, wedemonstrated that, by sharing entanglement between local andremote sites, secure quantum remote sensing is possible, and the
secure level is characterized by asymmetric Fisher information gain.
Concretely, only the local site can acquire the estimated parameter
accurately with Fisher information approaching 1. In contrast, theaccessible Fisher information for an eavesdropper is nearly zero even
if he or she obtains the raw sensing data at the remote site. ThisNTT Technical Review

Vol. 18 No. 9 Sept. 2020

achievement is primarily due to the nonlocal calibration and controlof the probe state at the remote site. Our results explore one significant advantage of “quantumness” and extend the notion of quantum
metrology to the security realm.

Determination of Frequency Response of MEMS Microphone from Sound-field Measurements Using Optical
Phase-shifting Interferometry Method
D. Hermawanto, K. Ishikawa, K. Yatabe, and Y. Oikawa
Applied Acoustics, Vol. 170, No. 15, 107523, July 2020.
Accurate determination of microphone sensitivity is important tobuild reliable acoustical instruments. Sensitivity is usually determined by calibration. However, because current microphone-calibration methods determine sensitivity from a mathematical model
derived from the geometry of a conventional condenser microphone,they cannot be applied to the calibration of a microelectromechanicalsystems (MEMS) microphone straightforwardly. To compensate forthis geometry difference with current calibration methods, someresearchers proposed the development of adapters that fit the conventional calibration apparatus and modified the calibration procedure.
In this paper, we propose a different approach to calibrate a MEMS
microphone. Sensitivity is calculated directly from the measurementof the sound field applied to the MEMS microphone and its output
voltage. The projection of the sound field is measured by parallel
phase-shifting interferometry (PPSI), and sound pressure on the
MEMS microphone is obtained by tomographic reconstruction.
Experimental calibration of a MEMS microphone was conducted andvalidated using a microphone-substitution method to evaluate thediscrepancies in the sensitivity results. It is shown that the proposed
method can be used to determine the frequency response of a MEMS
microphone in the frequency range of 1000 to 12000 Hz.

55