NTT Technical Review

October 2022 Vol. 20 No. 10
View from the TopYoshiaki Maeda, Senior Executive Vice President, NTT DOCOMO

Front-line Researchers

Shingo Tsukada, NTT Fellow, Bio-Medical Informatics Research Center, NTT Basic ResearchLaboratories

Rising ResearchersArinobu Niijima, Distinguished Researcher, NTT Human Informatics Laboratories

Feature Articles: Research and Development toward a Well-being SocietySupporting a Well-being Society that Integrates the Digital and Real Worlds

Social Well-being: Connections that Allow Altruistic Coexistence of Individual Autonomy andCollective Harmony

Healthcare Science for Lifelong Health—Visualizing Biological Rhythms and Regulating
Them on One’s Own



Human Digital Twins for Well-being

Human Information Science Research for Understanding and Improving Our Well-being

Feature Articles: Adapting to the Changing Present and Creating a Sustainable FutureCommunication Science that Adapts to the Changing Present and Creates a Sustainable
Future—Aiming to Create Technology that Brings Harmony and Symbiosis among People,
Society, and the Environment

AI Hears Your Voice as if It Were Right Next to You—Audio Processing Framework for
Separating Distant Sounds with Close-microphone QualityPersonal Heart Modeling Using Mobile SensingSmart Traffic Coordination via Learnable Digital Twins—Future Possibilities of Distributed
Deep Learning

Number Theory and Quantum Physics Based on Symmetry—Themes from Quantum Optics

Regular Articles
Development of a Rust Removal Tool Using a Diffractive Optical Element and High-power
Laser and Its Application to Asbestos-melting Tools

Global Standardization Activities
Standardization Activities for Optical Fiber and Cable Technology in International
Electrotechnical Commission

Practical Field Information about Telecommunication Technologies
Case Study of Audible Noise on Analog Leased Lines Caused by Electromagnetic Disturbance

External Awards

View from the Top

I Want to Create New Value andNurture It until It Becomes
Commonplace in Society
Yoshiaki Maeda
Senior Executive Vice President,
NTT DOCOMO
Abstract

On January 1, 2022, NTT Communications and NTT COMWARE
became subsidiaries of NTT DOCOMO, starting the new DOCOMOGroup. Under the brand slogan, “Changing Worlds with You,” the new
DOCOMO Group is working together with its customers and partners tocreate innovations and bring changes to society. We interviewed Yoshiaki
Maeda, senior executive vice president of NTT DOCOMO, about strategiesof the group’s smart life business and his attitude toward work.
Keywords: smart life business, social OS, purpose and values

The growth strategies of the new DOCOMOGroup and its smart life business—Congratulations on your appointment as senior
executive vice president. First, could you tell us about
the management condition and strategies of the new
DOCOMO Group?
Thank you very much. As senior executive vicepresident, my responsibilities have greatly expandedin addition to those of my previous position as executive general manager of the Marketing Platform
Division. Since assuming my new position, my passion for resolving social issues has been increasing,which is energizing me.
On January 1, 2022, NTT Communications and
NTT COMWARE became subsidiaries of NTT
DOCOMO, forming the new DOCOMO Group. On
July 1, 2022, NTT DOCOMO celebrated its 30th year
in business. In this milestone year, the new DOCOMOGroup integrated enterprise and consumer businesses
NTT Technical Review

Vol. 20 No. 10 Oct. 2022of the three companies according to function and has
made a full-fledged start with the following three segments as pillars of its business. Focusing on mobilephone services and expanding new services such as
fifth-generation mobile communications system (5G)
and 5G SA (stand-alone network), (i) the telecommunications business involves reforming sales channels and network structures; (ii) the smart life business aims to increase revenue by more than 120-billion yen year-on-year by accelerating businessgrowth in areas such as finance, payments, and video/
entertainment; and (iii) the enterprise business aims
to increase revenue by 55-billion yen year-on-year byrenovating our portfolio with new services, including
the expansion of cloud solutions and by providing
one-stop services that integrate mobile, fixed-line,
and cloud services. I’m in charge of the smart life
business.
NTT DOCOMO launched its 5G services in March
2020, and has been working on implementing 6G.
The implementation of 6G is one of the key project1

View from the Topthemes concerning the NTT Group’s Innovative
Optical and Wireless Network (IOWN), and we areaiming to fuse 6G and IOWN technologies under thetitle “5G Evolution & 6G powered by IOWN.” In the“DOCOMO 6G White Paper,” we reported that wehave developed a platform for enabling human augmentation, which is a new value we will be offeringin the 6G era to enhance human senses through networks. This is the world’s first platform for humanaugmentation.
In accordance with NTT Group’s environmental
and energy vision “NTT Green Innovation toward
2040,” NTT DOCOMO has positioned sustainability
at the center of its management. Recognizing thattackling climate change is an important corporate
issue, we are working to reduce greenhouse-gas
emissions from our business activities to zero by2030. We joined the Science Based Target initiative(SBTi), which aims to limit global temperature rise
above pre-industrial levels to 1.5°C, in February
2022. We will promote faster communications andpower saving by developing technologies that contribute to reducing greenhouse-gas emissions in nextgeneration networks and information-processing
infrastructure. We will also increase the renewableenergy use ratio of electricity consumed by our business activities to 100%, including the purchase ofnon-fossil fuels certificates designated as renewableenergy.
—What initiatives will be undertaken in the smart life
business?
We have introduced an in-house company systemand established the Smart Life Company. An inhouse company system is an approach to corporate
management that treats divisions within a company
as independent companies. By increasing the independence of our smart life business, we can makespeedy decisions on agile investment and hiring andtraining professional human resources, which will
drive the evolution of our society at an accelerated
pace. At the Smart Life Company, we hope to create
new value for our customers by connecting multiple
partners, delivering that value, and nurturing it until it
becomes commonplace in society.
We will first refine the core assets of NTT
DOCOMO. For example, our points-based customer
loyalty program, “d POINT CLUB,” has approximately 90.4-million members. It is by far the largest
program of its kind in Japan. That fact should indicate
our reliability and stability. What’s more, approxiNTT Technical Review

Vol. 20 No. 10 Oct. 2022

mately 590,000 partner companies have been nurtured through d POINTs and financial settlements,
and the number of d POINT-participating stores hasreached 750. We also possess data and technologies
such as profiling AI (artificial intelligence), Mobile
Spatial Statistics (population statistics using mobile
network data), and authentication infrastructure thatform the foundation of our various services.
By polishing up these core assets of NTT DOCOMO
which support the growth of the marketing domainsof our partner companies, we will build a foundation
for promoting co-creation and alliances with partnercompanies, create and provide a group of services
that will enrich society, and grow together with ourpartner companies. In other words, I believe that thisis an effort to convert a customer base into a social
operating system (OS) to build a platform for transforming society as well as individual lives. This effort
involves not only finance, which is our strength, butalso other fields that we want to grow and expand,e.g., medical and healthcare, energy, and urban development. In the future, we want to construct a social
OS that can provide value from the entire NTT Groupin a one-stop-shop manner by using individual identifiers.

2View from the Top

I’m with NTT DOCOMO over 20 years and still
enjoy working here, never get tired of it—The idea of constructing a social OS shows thespirit of NTT DOCOMO.
At the Smart Life Company, when we set our goal
to work for the betterment of society, we rethought
our reason to exist, i.e., our purpose, and our values.
We believed that by clarifying and sharing our purpose and values, we could promote corporate growthand move in a positive direction. While we were making these decisions, we kept thinking “who we are
and what are we for?” Therefore, we traced our history back to the Nippon Telegraph and Telephone
Public Corporation, which was one of the former“three public corporations and five government enterprises” in Japan, and we concluded that our origins lie
in the desire to improve Japan during that era. We also
reaffirmed that we have always been passionate about
making communications and connections between
people as well as social evolution a commonplace.
As exemplified by our i-mode (mobile Internet service), by using computing and networks like twowheels of a bicycle, NTT DOCOMO has been a pioneer in providing new value for about 20 years.
Across Japan, many people rely on the infrastructurethat we provide, and we take pride in our responsibil-

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

ity and capability to provide it. Judging from theseachievements, I believe that we have continued toembody “social-value creation.” When we launched
the Smart Life Company, we clearly stated our purpose is “to connect and nurture until it becomes thecommonplace of tomorrow.” In accordance with that
purpose, we will continue to create and nurture newvalue in Japan and around the world in the same fashion as we have been doing in cooperation with ourpartner companies.
We also herald the three values that should beshared to fulfill our purpose, i.e., “There is no end to
the pursuit of providing better value,” “Think big, act
fast,” and “The growth of society begins with thegrowth of oneself.” Although there may be days ofhardship in the midst of competition, I want to keep
challenging myself and remain unwavering toachieve personal growth.
—Your company’s purpose and values give us hopefor a bright future. How did you develop your powerful and positive attitude?
I feel that the experiences I have had since I started
working, and the impact they have had on me havemade me the person I am today. When I decided tomove from Recruit Holdings to NTT DOCOMO
about 20 years ago, my colleagues at the time worried
about me saying, “Are you going to be okay?”
Despite their worries, I was excited about working for
NTT DOCOMO. The reason for my enthusiasm was
the fact that at that time NTT DOCOMO had alreadyprovided services to tens of millions of customers andhad the dynamism to listen to responses from them.
At the same time, my mentor, Takeshi Natsuno, thecreator of i-mode, often said, “What are you doing
this job for? I do this job to make society more convenient, enjoyable, and affluent.” I felt a great deal of
empathy for his words, and I was proud of what I wasdoing when I first joined the company. From that timeon, I myself had a strong desire to realize that my life
was changing and becoming more interesting.
It’s been more than 20 years since I started working
at NTT DOCOMO, and I’ve never gotten tired of mywork. Over the last 20 years, the world has undergone
various evolutions, and each time we have been
involved in those evolutions, we have also evolved
while creating new things. I feel strongly that I havebeen able to grow and develop myself as NTT
DOCOMO has been evolving and receiving attention
from society. It has been an opportunity given to me
by NTT DOCOMO for which I am grateful. That’s
3View from the Top

why I want us to be the kind of company that makes
people think, “Wow, DOCOMO!”
Never do things by halves—I can feel your love for NTT DOCOMO and yourpassion for your work. Can you tell us what is important to you in your work?
It is said that people who change the world are outsiders, young people, and idiots, and when I joined
NTT DOCOMO, I was assigned to a department towhich many people had moved in from other companies. Of course, there were also people who had beenwith NTT DOCOMO and they communicated wellwith me, and I feel that the trust we built while working together is rock-solid. Based on these experiences, I have two key points I want to emphasize as a top
executive. First, I want to run a business that makes
everyone who works in it can feel positive possibilities. I want to and must operate the business andpractice management that can embody the ideal
image of NTT DOCOMO in terms of what sort ofpresence we can continue to have in society. Second,
to manifest the first point, I want all our employees totake responsibility and have the initiative to carrythrough their work. I myself will always try to communicate with them so that everyone thinks in thatway.
Again, for the reasons mentioned above, our purpose and the values to be shared are put in writing.
For example, we have listed a series of important
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

messages we want to communicate, such as always
having a deep understanding of our customers, focusing on the value for each customer and partner, andmeeting everyone’s expectations with sincerity andintegrity. These messages emphasize the importanceof our approach of “Never do things by halves.”
There will be many times when you are working
that you will think, “That’s about right.” However, atthat moment, your growth stops. I believe that those
who are willing to go the extra mile will drive society
and ultimately be the winners. In fact, I’m still in theprocess of growing, and it is difficult to stay motivated all the time. Even so, I’m inspired by thosewritten messages that I keep on my desk. I haven’t
felt this enthusiasm for a long time.
—I’m sure that your employees also feel their enthusiasm. How does research and development play arole in embodying such enthusiasm?
I have high expectations for research and development. The construction of a social OS using our coreassets can be done through the culmination of thewisdom of everyone in research and development.
The final form of this system is a social OS that can
provide all NTT Group services. I look forward to
taking on the challenge of flexibly and promptly
addressing the needs of our partners while keeping
abreast of social trends by harnessing our assets,
namely, the communications technology that supports the network, devices connected to the network,and applications that run on those devices.
4View from the Top

I also want us to develop technologies that can provide interesting and enjoyable experiences andworldviews, which mainly concern the entertainment
field. I want to use such technologies to create newservices in collaboration with our partner companies.
Incidentally, Japan’s first “smart arena,” scheduled toopen in Aichi Prefecture in the summer of 2025, is achallenge to create an unprecedented experience ofentertainment by introducing cutting-edge technologies. As well as making customers happy in thatarena, I want to link that service to new business.
Thus, let’s do our best to create impactful, sensational, and dramatic experiences.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Interviewee profile
Yoshiaki Maeda joined NTT DOCOMO, Inc. in
2000. He became director of the Consumer
Services Department in 2008, senior vice president and general manager of the Platform
Business Department in 2017, and executive vicepresident and executive general manager of theMarketing Platform Division in 2020. He has
been in his current position since June 2022.

5

Front-line Researchers

I Want to Reduce the Sudden
Misfortune Caused by Heart Failures
by Bridging the Gap between
Medicine and ICT
Shingo Tsukada
NTT Fellow, Bio-Medical Informatics
Research Center, NTT Basic ResearchLaboratories
Abstract
Electrocardiograms (ECGs) are widely used in medical institutions for diagnosis, vital monitoring, health checkups, and automated external defibrillators. With the development of information and communication technology (ICT) and advances in
information-processing technologies such as machine learning,
the application of ECGs is expanding to self-care and other fields.
In today’s aging society, the need for in-home medical care andtelemedicine using ECGs is rapidly increasing due to the increasein the occurrence of cardiac diseases. We interviewed Dr. Shingo Tsukada, an NTT Fellow who has been
conducting interdisciplinary research that integrates medicine and ICT by drawing on his clinical experience as a physician.
Keywords: tensor ECG, hitoeTM, interdisciplinary research

Wearable 3D ECG device with new lead system
and analysis method: tensor ECG—It has been three years since our last interview.
How are your research activities going?
Things are going well. Using our experience indeveloping a wearable electrocardiogram (ECG)
device using hitoe TM and by combining medical
knowledge of clinical ECGs and recent informationprocessing technology, we are currently constructinga system for constantly measuring and analyzing
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

ECGs. Introduced in 2014, hitoeTM is a functional
fabric made by coating a conductive polymer on a
cutting-edge fiber material called nanofiber fabricfrom Toray Industries, Inc. A clothing-type vital sensor using hitoeTM can measure biosignals with highsensitivity without burden on the wearer.
In the process of our research on hitoeTM, we werealways conscious of ensuring our research results
would be practical. The major international sporting
event held in Tokyo in 2021 gave us an excellent
opportunity to do just that. We have been involved in
conditioning athletes taking part in swimming and6

Front-line Researchers

cycling competitions by using the advantages ofclothing-type hitoeTM capable of sensing vital data
during exercise.
To support two swimmers, Takeshi Kawamoto(sponsored by Toyota Motor Corporation) and Ai
Soma (sponsored by Miki House), whose trainingbase is the swimming club of Chukyo University, we
proposed an exercise program for countering theirdecline in motor function (centered on the thorax)
due to chronic muscle tension associated with theirlong-term and intensive training. This program promotes awareness of the functional coordination of thespine, ribs, and core muscles in a manner that restores
natural and efficient body movement.
In training, changes in the swimmers’ streamlined
postures before and after participating in our exercise
program were evaluated by photographing them fromthree directions and evaluating ease of breathing, lifting of the arms, and expansion and contraction of thechest circumference during breathing using a multisensor belt equipped with hitoeTM that measures myoelectricity, respiration, and motion. Due to the
COVID-19 pandemic, we provided coaching for athlete conditioning via a smartphone-based web-conferencing system. Both Takeshi Kawamoto and Ai
Soma were successful at the Japan National Championships, and I am glad that we could continue thisinitiative despite the pandemic.
We have been involved in the measurement, analysis, and visualization of surface myoelectric potentials for cycling. Using hitoeTM as a bioelectrode, we
collaborated with Bridgestone Cycle Corporation and
NTT DATA to evaluate pedaling from the perspectiveof muscle fatigue and muscle activity in top Japanese
athletes such as Eiya Hashimoto of Team Bridgestone
Cycling. The athletes’ surface myoelectric potentials
were measured at various locations, including thecompetition venue for the major international sporting event held in Tokyo, and the pedaling characteristics of each athlete were visualized using the collected measurement data. The visualized characteristics
were then fed back to the athletes for discussions
based on their intuitions and subjective opinions in amanner that enabled us to identify points for strengthening.
Knowing the status of muscle activity during competition is an important factor in improving and conditioning an athlete’s performance. The surface myoelectric potentials measured using hitoeTM can monitor muscle activity from the surface of the body, thus
are considered effective for on-site analysis of exercise because they can be measured by having the user
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

simply change clothes in a manner that imposes little
burden on the body. Through the above-described
initiatives for cycling, we confirmed the usefulness of
hitoeTM as a bioelectrode.
—New research themes derived from hitoeTM havebeen created, right?
Having previously worked as a physician, I joined
NTT mid-career in 2010 and started my research in a
new field almost from scratch. I was in charge ofeverything from the invention of bioelectrodes to thecreation of fabrics for bioelectrodes that enable constant monitoring of heartbeats and ECGs just by
wearing it and implementation of these technologies.
Although we have proposed various bioelectrodes
and wearable devices, only a small percentage havebeen implemented, and other companies are in thesame situation. Fortunately, my proposal was a new
type of a wearable biometric sensor, so it attracted
attention in Japan and from abroad, and I was busy
flying all over the world to promote it. In the midst of
the stress brought on by that travelling, and because
doctors often neglect their own health, I suddenly suffered a heart attack. I was shocked by the ECG
obtained using the wearable ECG device that I was
studying at the time and by the ECG obtained during
the provocation test conducted for precise examination at the hospital. Compared with the intense subjective symptoms I experienced during the heart
attack, the abnormality (signal waveform distortion)
shown by the ECGs were extremely small.
Regardless of the fact that wearable ECG devices
have been studied with the aim of avoiding heart
attacks and enabling early diagnosis of heart failure,
conventional methods of interpreting an ECG (which
are the criteria for determining normal or abnormal
ECG) have their limitations. I became aware of theexistence of small abnormalities that could be overlooked and cause heart attacks even when using thesewearable ECG devices. Overcoming this limitation
will require innovative techniques for ECG analysis
by using information-processing technology. The
behavioral restrictions placed on people due to the
COVID-19 pandemic allowed me to change my busy
life and take time to focus on this issue and create
tensor ECG, a new method of analyzing ECGs.

7

Front-line Researchers

Mediastinum(large blood vessel)
Right lungLeft lungY
Heart
Z
Apex of the heart
Anterior left
ventricular wall

X(a) New lead system* for obtaining ECGs using the apexof the heart (tip of the heart) as a reference point(b) Belt-type tensor ECG using
hitoe™ electrodes*Position and combination of electrodes for measuring ECG(heart generated potential)

Fig. 1. Tensor ECG.

The experience of a clinician and non-specialist
in information processing and statistics
was useful—Your unfortunate experience was behind the development of a new method for ECG analysis. Could youtell us more about tensor ECG?
An ECG is a time-varying representation of potential differences acquired from multiple bioelectrodes
placed on the chest, extremities, and other parts of the
body. The chest electrodes are less affected by body
movement, thus produce relatively large cardiac
potentials. However, the other electrodes are significantly affected by body movements; accordingly,
ECG measurements are basically conducted with theuser in a resting state.
To find minute abnormalities in an ECG (i.e., distortion of the waveform), it is necessary to stably
record the ECG over a long period. We therefore
devised a wearable ECG device with a new lead system, i.e., placing electrodes in three linearly independent directions using the apex region of the heart (i.e.,from the apex to anterior left ventricular wall), where
the heart is closest to the rib cage, as a reference point(Fig. 1). Since the heart and its movement are threedimensional (3D), a 3D ECG corresponding to the
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

movement of the heart is obtained from electrodes
placed three-dimensionally around the heart. To
improve wearability of the electrodes and minimize
the effects of body movement on the position of theelectrodes, the electrodes and wiring are integrated inan elastic belt, which can be easily fitted to the user’s
body by simply tightening the shoulder and waist
belts. We also developed a polygraph (mechanocardiogram) that simultaneously measures cardiac output and deep-vascular pulse waves.
Conventional criteria for normal ECG include awide range of values for waveforms and intervals,
and individual criteria are currently used for eachdisease or pattern. Even when a person has a heart
attack (angina pectoris or other abnormality), theirwaveform may only be slightly distorted. A methodfor quantitatively evaluating abnormalities in atypical
ECGs has not been developed, and in some cases,
abnormalities have been judged to be normal. An
ECG is a recording made from the body surface ofelectrical potentials generated by the excitation(called action potentials) of numerous myocardial
cells. The action potential of myocardial cells cannot
be measured from the body surface. Therefore, we
devised a method of (i) statistically modeling the timing of changes in the action potential by using a
Gaussian distribution and (ii) estimating the collective
8

Potential (µV)

Front-line Researchers(a) Normal ECG (left) and action potential of the
myocardium (electrical excitation)

Normal ECG

Acute myocardial
infarction

Long-QT syndrome

Anginapectoris

Early-repolarization
syndrome

Time

NegativeT-wave(c) Analysis of ECG on the basis of distortion ofaction potential of the myocardium**Tensor ECG is at the research phase, thus cannot be used for medical diagnosis.

(b) Abnormal ECG (red) and distortion of
myocardial action potential

Fig. 2. Visualization of cardiac abnormalities by using tensor ECG.

transition of action potentials of the myocardial cells(cardiac muscle) from an ECG. This method makes it
possible to resolve, amplify, and visualize atypicaldistortions in an ECG (Fig. 2).
We are currently verifying whether the parameters
obtained with tensor ECG are effective indicators forclassifying complex cardiac abnormalities and quantifying and evaluating minute distortions in ECGs
that have been hitherto overlooked in a unified manner. We hope that this method will be useful for diagnosing arrhythmias (irregular heartbeat) associatedwith heart failure, ischemic heart disease, and sudden
cardiac death.
—Has this type of analysis of ECGs been done
before?
I have checked previous papers and prior patents
regarding tensor ECG, but I found no similar cases,
so I believe the novelty of this development is high.
In Europe and the US, where heart failure and heart
attacks are the leading causes of death, research inthis field is thriving, and competition in research anddevelopment has been intensifying as Apple andother companies enter the field. Many of theseresearch efforts use machine learning and artificial
intelligence (AI) to automatically determine abnorNTT Technical Review

Vol. 20 No. 10 Oct. 2022

mal ECGs and attempt to detect abnormalities that
could not be detected before. However, ECG interpretation using AI faces certain issues, for example,
the dependence of the ECG data used to train the AI,
ambiguity of causal relationships between ECG dataand judgement, and necessity of preparing a large
amount of ECG data. The so-called inverse problem
in electrocardiography, which involves calculating
myocardial action potentials from ECGs and analyzing abnormalities, has been regarded as a difficult
problem with no solution. Moreover, abnormalities in
ECGs often involve only slight distortions of shape orchanges in potential.
However, I know that certain clinicians are good atreading subtle changes in ECGs. I hypothesized thatthey identify abnormalities on the basis of their experience of examining ECGs of many patients. To solve
the above problem, I studied information processing
and statistics. The medical science that I studied atuniversity was a far cry from today’s information
processing, so I had to start from scratch again. However, one day, while relearning about the Gaussian
distribution in statistics, I had an inspiration of a very
simple model equation and found that it could solve
the problem, albeit with some limitations. Perhaps it
was my amateur ideas concerning information processing and statistics that worked.
9Front-line Researchers

The researcher’s mission is to find and nurture
new seeds for a new era—In our last interview, you mentioned that yourdream is to develop advanced medical devices for
detecting signs of illness, and your dream looks as if
it will be coming true very soon.
I do not currently know if tensor ECG is the correct
method or useful for clinical medicine. If this method
is validated, it would truly be a dream come true aswell as a major breakthrough. We are now entering
the stress-test phase, and we face a number of technical challenges and must overcome a number of major
hurdles before we can start implementing it. We havejust started clinical research with tensor ECG, but if
its effectiveness and value are recognized, it could be
adopted as a method for analyzing ECGs in about
three to five years. However, AI diagnosis of ECGs isdeveloping at a breakneck pace, so it is possible that
tensor ECG will lose out. Tensor ECG is a versatile
analysis method applicable to all types of ECGs. If its
clinical efficacy is confirmed, it could contribute tosociety by enabling precise analysis of a person’s
ECGs and detection of signs of heart disease by simply clicking on a website in, say, ten years or so.
Academically speaking, I’m considering submitting papers to interdisciplinary journals that cover
topics between information processing and medicine.
I realize that many of today’s research problems cannot be solved by research in a single field of expertise.
In fact, in Europe and the US, cross-disciplines ordouble majors are common—that is, researchers andspecialists who have mastered one field of expertise
pursue another field of expertise—or work closelywith those with another field of expertise to solve
problems. However, I feel that in Japan, cross-disciplinary and interdisciplinary studies have not progressed, and such researchers of my generation are
few and far between.
My work as a researcher at NTT spans the fields of
medicine, physiology, medical engineering, and bioinstrumentation and information processing. I hopeto help build a bridge among experts in different
fields and address social issues such as rising medical
costs.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

—What do you value as a researcher?
I believe that the role of a researcher is to find newseeds and nurture them for the next generation. In
every field, there are research themes that attract
attention at certain times, but not all are necessarily
connected to important issues. Therefore, I want toapproach my research with an eye to the importanceof the theme and the social necessity of research onthat theme. While it is essential to enhance one’s
expertise, I also want to actively engage in interdisciplinary exchanges and hands-on experience. Throughvarious exchanges, I can understand the difficulties
and technical bottlenecks in each field, and unique
ideas can be born from them. To facilitate such interaction and communication, I hope to continue to be a“handyman” who is easy to talk to, as I said in our lastinterview.
From my experience as a physician dealing withpatients, I believe that happiness is not simply
brought by success, achievement, status, and so on;rather, by being blessed with family, friends, and colleagues and interacting with others. This feeling is
one that I share with many of the health-care professionals I have worked with. It seems that researchers
who have good relationships with those around them
are more likely to be able to continue their research
activities and achieve more consistent results. Of
course, some researchers have achieved remarkable
results in their fields of expertise on their own.
Research styles can vary individually, but I’d like tocontinue my research activities while engaging witha variety of people and helping one another.
Researchers are evaluated on the basis of the number of papers they have published, the impact factorof those publications, and other indicators, and I feelthat what researchers who have achieved outstanding
results have in common is being good communicators. That means they have outstanding communication skills and are well versed in fields outside theirexpertise; for example, they can relate to topics anddifficulties in other fields. I hope to be the same way;that is, I want to continue to conduct research that willlead to a brighter future in a manner that enables myresearch results to be as much use to society as possible and allows young researchers to have more
opportunities to be active.

10

Front-line Researchers

Interviewee profile
Shingo Tsukada graduated from Toyama
University School of Medicine and received a
medical license in 1990. He also received a Ph.D.
in medicine from the University of Tsukuba in
2003. He was a visiting researcher at the
University of California San Diego from 2003 to
2005. He joined NTT Basic Research Laboratories
in 2010 as a research specialist. He has been
studying cardio vascular function and neuronal
regulation. His current interests include the detection of biomedical signals and functional modification using novel wearable-type and implanttype bioelectrodes based on the composites ofconductive polymers with various fibers and textiles. He is an inventor of the textile bioelectrodehitoeTM. He is a member of the PhysiologicalSociety of Japan, the Japan Society of Applied
Physics, the Japanese Circulation Society, theJapanese Orthopedic Association, and theJapanese Association of Rehabilitation Medicine.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

11

Rising Researchers

Muscle Interface Technology for
Augmenting Human Capabilities
through Efficient Motor Skill
Learning by Electrical Muscle
Stimulation
Arinobu Niijima
Distinguished Researcher, NTT Human
Informatics Laboratories
Abstract
Conventional motor skill learning has been centered on thevisual and auditory senses, but for beginners, acquiring the correct
way of using one’s muscles like experts takes time. In thisinterview, we talked with NTT Distinguished Researcher Arinobu
Niijima about muscle interface technology for achieving efficient
motor skill learning using electrical muscle stimulation.
Keywords: muscle interface technology, motor skill learning,
electrical muscle stimulation

Muscle interface technology for augmenting
human capabilities through efficient motor
skill learning—Dr. Niijima, what exactly is “muscle interface
technology for motor skill learning?”
In a few words, “muscle interface technology for
motor skill learning” is a system that supports motor
skill learning via muscles in sports, playing a musicalinstrument, and other endeavors. In the research that
I’m now engaged in, I’m targeting piano playing. On
analyzing the movements made by beginners andexperts with sensors, the data obtained shows that thephysical movements they make when playing the
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

piano are completely different from each other. One
reason why beginners cannot play the piano likeexperts is that they apply more force than necessary
or fail to use their muscles in the right way. With thisbeing the case, I thought about developing a systemthat, based on these physical differences between
beginners and experts, could enable beginners to
directly experience the movements of experts byapplying electrical stimulation to their muscles.
For example, conventional motor skill learning has
been centered on the visual and auditory senses inwhich an expert may verbally teach beginners how tomove their bodies correctly or beginners may simply
observe an expert’s performance. However, learning
an expert’s muscle movements by only visual or12

Rising Researchers

Electromyography sensors
1. Extensor pollicis longus
2. Extensor digitorum

Adjustable parameters
using a MIDI controller
1. EMS ON/OFF
2. EMS frequency
3. EMS stimulation time

Electrodes for EMS use
1. Supinator muscle
2. Pronator teres muscle

MIDI: Musical Instrument Digital Interface

Fig. 1. Example of muscle interface technology for motor skill learning in piano playing.

auditory means can be difficult. The aim of muscle
interface technology is to enable efficient learning ofcorrect muscle movement by moving a user’s muscles
in a more direct manner.
While at university, I played Sports Chanbara andcame to teach newly enrolled college students andeven elementary school students. It was at that timethat I learned that what I first had to teach all
beginners was “relax when you swing your sword.”
Additionally, when I was learning how to swim, theinstructor said the same thing: “Please relax.”Moreover, when studying how to play the piano froma book, the instruction “Try to relax when playing!”
was likewise included. I thought it was interestingthat “please relax” was a key point in learning justabout any movement. So having some interest in
motor skill learning, I began my research thinking, “I
wonder if motor skill learning could be supported bycomputer.”—What kind of method do you use to implement
muscle interface technology?
In this muscle interface, we use a technology called
electrical muscle stimulation (EMS) to move muscles
by electrical stimulation to teach beginners how
experts move their muscles. In this regard, a muscle
normally contracts by receiving an electrical signal
from the brain, but EMS applies electrical stimulationfrom an external source to the muscle instead of anelectrical signal from the brain.
This technology has been traditionally used in the
medical field for rehabilitation purposes, but I began
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

my research thinking that EMS has probably neverbeen used as an interface for healthy individuals tolearn motor skills. Let me describe how motor skill
learning in piano playing can be supported by EMS
using Fig. 1. In piano playing, there is a techniquecalled “tremolo” in which the player rapidly alternates
between two or more keys. For a beginner, thistechnique is seemingly easy but actually difficult, andas a result, a beginner get tired easily, preventing him
or her from playing the piano over an extended periodof time. To investigate the reason for this, we usedelectromyography sensors to analyze how beginners
and experts use their muscles while playing the piano.
We found that beginners apply more force thannecessary to the extensor digitorum and extensor
pollicis longus muscles when moving their fingers
compared with experts. In contrast, experts use theirsupinator and pronator teres muscles for wrist
rotation instead of using muscles to move theirfingers. As a result, they do not get tired easily andcan play the piano for a longer period of time. Tochange the way that beginners use their arm muscles,
we created a system that teaches beginners how toplay a tremolo while rotating their wrist like experts
by applying EMS to the supinator and pronator teres
muscles alternately. By practicing with this system, abeginner becomes increasingly aware of wrist
rotation instead of finger movements, and with a little
practice, learns the knack of playing tremolos for an
extended period of time without fatigue.
The key point here is that the beginner moves his or
her hand in time with the motion of the electricalstimulation thereby learning in an active manner. In
13

Rising Researchers

Reproduce motor skills (transcribe)
Biologicalsignals

Transcription
source(1) Obtain biologicalsignals, generate
motor model(2) Generate andpresent sensory
stimulation(4) Adjust
sensory
stimulation(3) Checkmotor state

Present
stimulation

Transcription
destination
Sensing ofoperation results

Fig. 2. Outlook for muscle interface technology linked with a human augmentation platform.

this way, motor skill learning progresses, and after
practicing, the beginner can play tremolos without
EMS using wrist rotation in the right way.
In conventional EMS technology used in humaninterfaces, there have been many studies on contracting
and moving muscles by EMS but there has beenpractically no research on reducing force applied to amuscle. In contrast, our muscle interface technology
indirectly transcribes a relaxed state of specificmuscles by applying electrical stimulation to a
beginner’s muscles based on data obtained on
efficient body movement by experts.
The most difficult thing about this research isdetermining the extent to which complicatedmovements can be taught. For example, a full-body
motion like a golf swing involves multiple muscles,
so helping a beginner to learn a golf swing becomes
a complicated problem. In such a case, the crux ofthis research is determining whether expert
movements can be taught by applying electricalstimulation to certain muscles to produce certain
movements. To this end, we must first obtainknowledge from research fields dealing with physicalmovement and then collect an extensive amount of
information from all sorts of media in addition to theliterature. We can then proceed to extract the waysthat certain movements should be made common tothese sources of information. After this, we can turnbiological signals during movements into data using
electromyography sensors or motion capture andanalyze which muscles are used in what ways. The
method of analysis here is not only to look at
individual muscle data. A technique called “muscle
synergy analysis” can also be used to analyze muscle
coordination, that is, to analyze which muscles
cooperate with which other muscles. By collecting
diverse types of data in this way, we can determine
which muscles we should focus on to reproduce
expert movements. In addition, we don’t stop at thisstep of collecting data. In the end, it is crucial that wecreate a system for actually applying electricalstimulation and try it ourselves, and that we conductsubjective evaluations on whether this system will
enable users to experience the feeling of motor skill
learning.
Future of muscle interface technology inbreaking down current barriers—What is your outlook for muscle interface
technology in the years to come?
Figure 2 shows how cybernetics technology suchas muscle interface technology that I am researching
can be linked to a human augmentation platform forcontrolling human movement via the network. As afuture outlook, I believe that it will be possible tosupport motor skill learning remotely in real time byusing movement transcription technology that copies
human movement to another person. This technology
transmits data on an expert’s body to a beginner at aremote location so that the beginner can receive

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

14

Rising Researchers

stimulation by electrical stimulation equipment.
In addition, I think it would be great if muscle
interface technology could be used to extend healthy
life expectancy. That is, when a person’s muscularstrength and cognitive abilities begin to decline withage and what he or she could do when younger can nolonger be done, muscle interface technology could be
used to synchronize past and present movements bydownloading old data and restoring the way that one
used to move. When a person stops trying out new
things, that person’s lust for life will quickly be lost.
Supporting one’s learning through a muscle interface
can overcome the first barrier to skill acquisition andenable people of any age to take up the challenge of
doing new things. In this way, I would like to help
people with their lives.
—Dr. Niijima, please leave us with a message forother researchers, students, and business partners.
I currently belong to NTT Human Informatics
Laboratories, which, as the name implies, is focusedon research centered about humans. Engineers andresearchers at times tend to focus their efforts on
improving technology while leaving people behind.
However, if one stops for a moment and considers theactual purpose of research, it is certainly to “improve
people’s lives and future.” As a researcher of humancomputer interaction, I myself treat humans as thetarget of my research, so I can identify with the visionof NTT Human Informatics Laboratories, which I
feel to be a very pleasant working environment.
A key strength of NTT is the ability to talk freely
with experts in diverse fields within the company. It
is said that my research field, human-computerinteraction, is a scientific field that is not defined by
specialized technology. When I began my research,
the knowledge and technology that I needed spanned
a wide range. For example, in motor skill learningthat I am now researching, not only do I need sensor
technology, analysis technology, and engineeringskills for combining a series of systems, I also needknowledge related to human movement from the fieldof neuroscience as well as information on various

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

types of muscles. In short, it is only by gathering suchknowledge and technologies from disparate fields
that I can research a single theme. At this time,surveying all of this information by myself would
take a considerable amount of time and labor, but at
NTT, it is easy to ask other researchers for assistanceor to hold meetings with them, so specialized
knowledge can be acquired in a relatively short time.
I also feel that the value of the NTT brand name andthe size of the company are amazing. It is often thecase that researchers fulfill their role by conducting
research, writing papers, and presenting themsomewhere at academic societies. At NTT, however,announcing new technologies developed for the goodof society draws a response several orders ofmagnitude greater than usual, which can lead to theactual implementation of those technologies insociety. In my opinion, only NTT can provide such asense of accomplishment and sense of worth.
I’ve recently noticed that there are young researchersand students that are conducting research while still
not knowing what kind of research they personallylike. What I have learned over the years in myresearch is that it’s important in the end to do what
you enjoy. To give an example, my research during
my university and graduate school days was like ashort-distance sprint—even if I wasn’t particularlyinterested in that research, it consisted of just shortterm objectives, so I gave it my best effort. On theother hand, once I entered adult life as an actual
employee, I could imagine my research continuing on
for several decades rather like a long-distance run,and I thought that it would be difficult to stick withany research that I had no real interest in. So on
thinking on how I could run for such a long distance,
I realized that doing something because you truly likedoing it could be a powerful driving force. Everyday,
there are a variety of things that catch my interest, andit often happens that I begin research on some topic
thinking, “This is really interesting!” Of course, adesire to solve social problems is important, but I also
think it’s important that the individual take a goodlook at what he or she inherently likes and to verbalizethat preference clearly.

15

Rising Researchers

Interviewee profile
Arinobu Niijima received his Ph.D. inengineering from the Graduate School of the
University of Tokyo in 2017. He entered NTT in2012 and is currently a member of NTT Human
Informatics Laboratories, where he has been a
distinguished researcher since 2021. He is
engaged in the research of informationcommunications and human interfaces/interactions. He is the recipient of the CyberSpace Research Award from the Virtual Reality
Society of Japan, the Best Poster Award at
Augmented Humans 2021, and other awards.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

16

Feature Articles: Research and Development toward a
Well-being Society

Supporting a Well-being Society that
Integrates the Digital and RealWorlds
Katsuhiko Suzuki and Masaru Miyamoto
Abstract

After reviewing the spread of digital technology in society that has accompanied the development of
information and communication technology, this article introduces a well-being digital-real fusion society that is necessary today, and the efforts being made by NTT laboratories to achieve it.
Keywords: well-being, digitization, interdisciplinary research

1. Well-being and digital society
1.1 Well-being as an integral subjective measure
The concept of well-being has been attracting muchattention, but why? Let me start by first discussing
what is actually meant by well-being. Well-beingrefers to a state of being in which people are satisfied
both physically and mentally [1]. The novel coronavirus (COVID-19) pandemic has adversely affected
the well-being of people and is one of the main reasons well-being is considered so important today.
The graph in Fig. 1 shows how a person’s contentment can change over time. We all experience goodtimes and bad times. We believe that well-being is not
something that can remedy a bad situation by simply
crossing a threshold nor something that is felt only inpassing moments but an integral subjective state ofmind that takes into account a person’s hardships andtroubles from the past to the present.
1.2 Well-being in a digital society
To determine what should be done in the information and communication field to promote well-being,
we should first examine the current situation regarding the impact that the digitalization of society has
had on well-being. Let us look back on the evolutionof this digital society in Japan, starting with thefounding of NTT’s predecessor, Nippon Telegraphand Telephone Public Corporation, in 1952, as shown
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

in Fig. 2.
During NTT’s early years, from the 1950s to 1980s,
fixed-line telephones gradually became widespread.
When people wanted to talk to someone far away,
they would have to go to the house or room where thephone was, and had to be mindful of the call charges
that accumulated as they made their call. It is thoughtthat telephones contributed greatly to the high economic growth of this period in terms of increased
productivity.
With the launch of Internet access services in 1996
and the availability of broadband fixed-line networks
due to the introduction of optical fiber technology in
2001, coupled with the launch of cellphone services
in 1987 and i-mode (NTT DOCOMO’s mobile internet service) in 1999, the period from the mid-1990s
through the 2000s saw the development of a network
environment that was available anytime, anywhere.
However, due to the lack of content, this period wasrecognized as an era when people connected to a network anytime, anywhere whenever necessary.
With the release of the iPhone in 2007 and Android
devices in 2008, smartphones and social networking
services became widespread, and now, everyone isalways connected to a network. Another majorchange was the widespread switchover to teleworking that was prompted by the COVID-19 pandemic in
2020.
It is important to note that the current era of17

Feature Articles

Happiness

Good

Time

Well-being
Bad
Medicine

Fig. 1. Well-being as an integral subjective measure.

Japanese
population

100 million (1966)
5.7%(1960)

Aging rate
Society

128 million (2008)

Population growth
7.1%(1970)

Population decline

9.1%(1980)

12.1%(1990)

Osaka Expo
(1970)

EventsJapanese
regnal era

17.4%(2000)

Great Hanshin-Awaji
Earthquake (1995)

23.0%(2010)
Great East Japan
Earthquake (2011)

Showa

Heisei1999

i-modeCellphones
Cll hones

Information
communication

1987
1968

Pagers
1996

1952

1985

Nippon Telegraph and Telephone Public Corporation

1950

1960

1970

1980

Smartphones

OpticalOti l fib
fiber

ISDN

Fixed-line telephones

Reiwa

Internet connection

2001

1988

NTT

28.9%
(2020)
COVID-19
pandemic(2020)

BubbleeconomyPost-war economic boom

Communication
services

NOW

1999

NTT
1990

NTT Group *Holding company structure
2000

2010

2020

Aging rate: ratio of the population aged 65 years and older to total population
ISDN: integrated services digital network

Fig. 2. The evolution of digital society in Japan.

constant connection to digital space is only about 10
years old, which is just a fleeting moment in the long
history of humankind. It is thought that this state ofbeing constantly connected to digital space for long
periods of time could have a significant impact on
people’s well-being, which is an integral subjective
measure based on long-term experience.
Another important factor is Japan’s population,
which had been increasing after a period of high economic growth, but peaked in 2008 and has since been
decreasing due to the declining birth rate and aging
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

population. Against this background, the structure ofJapanese society has changed. People now demand
new, flourishing lifestyles rather than ever-increasing
economic prosperity, and well-being is attracting
attention as a result.
1.3 What is required for well-being research
Based on this situation, the requirements ourresearch on well-being should focus on can be summarized with the following three points.
(1) An interdisciplinary approach18

Feature Articles(1) Social well-being

6. Self-transcendence

04: Self-transcendence

Altruistic coexistence of individual autonomy and collective
harmony centered on the digitization of society(3) Human digital twins

03: Individual

5. Desire for self-realization
4. Desire for approval

Support self-actualization andsensory communication basedon the digitization of people

02: Society/group

3. Social desire
2. Desire for safety

01: Life/body

1. Physiological desire(2) Lifelong health support

Support lifelong health by visualizing body rhythms basedon the digital representations of the human body(4) Human information science

Maslow’s hierarchy of needs

Understanding and intervening in the essential nature ofwell-being, centered on basic research of humans(a) The hierarchy of well-being(b) Initiatives of NTT laboratories

Fig. 3. The hierarchy of well-being and efforts being made by NTT laboratories.

The research must be interdisciplinary. The studyof well-being requires not only an engineering
approach based on measurement, analysis, and intervention in some form but also an academic approachthat includes diverse fields such as psychology, philosophy, ethics, law, and medicine. To achieve this, it
is essential to work with experts from universities in
addition to collaborating with NTT laboratories personnel having expertise in a wide range of fields.
(2)	The pursuit of possibilities through digitalization
Given that digitization has a major impact on wellbeing, research should be conducted to explore its
possibilities. Information and communication technology allows people to establish personalized connections in diverse ways over any distance and can be
expected to lead to the creation of new ways of living,working, and learning.
(3)	The elimination of obstacles with digital technology
Research is also needed to identify factors that
inhibit well-being due to digitization and to suppress
these factors. We should respect people’s privacy andavoid creating a surveillance society. We should also
avoid fueling disparities and divisions through fakenews and slander. It is important to bear in mind thenegative aspects of digitalization and consider countermeasures.
2. The hierarchy of well-being

cal structure of well-being since it is a very broadconcept (Fig. 3(a)). This hierarchy is an extension ofwhat Hiroi [2] described as a hierarchy of well-being
based on the hierarchy of needs proposed by theAmerican psychologist Abraham Maslow. Hiroi ismainly concerned with the role of public policies andprivate companies and the younger generation’s
awareness of social contribution, whereas in thisarticle, we specifically describe the hierarchy of wellbeing and its relation to information and communication.
Although we are aware that various taxonomies ofwell-being have been proposed, and that Maslow’s
hierarchy of needs theory has its critics, we will use
this taxonomy with priority given to ease of understanding and the ability to obtain an overall view ofvarious approaches to well-being, even for newcomers to the subject.
2.1 Life/body
The most fundamental aspect of well-being is thelife and body aspect, i.e., being physically healthy
and free from life-threatening conditions. Services
have been developed that use devices such as smart
watches to sense, analyze, and provide feedback on
the basis of vital data such as the wearer’s heart rate
and number of steps taken. Devices that can provide
more detailed information on the wearer’s physicalcondition are expected to become more widespread.
The characteristics of the human body are universal
and do not depend on individuals.

Before describing our research into well-being at
NTT laboratories, we will first explain the hierarchiNTT Technical Review

Vol. 20 No. 10 Oct. 2022

19

Feature Articles

2.2 Society/group
The next layer is the achievement of well-being
through face-to-face interaction with groups andsocial interactions with any number of other people.
The development of information and communication
technology has made it possible for more people tointeract anytime, anywhere, even when they arephysically distant from each other. It can also haveadverse effects such as social-networking fatigue
caused by prolonged connection and social fragmentation caused by slander and fake news.

peers (e.g., in a company or at school) can involve
sacrificing one’s individual goals, so there could be atendency for groups and individuals to become diametrically opposed. To solve this problem, we areresearching social well-being by defining it as a statein which individual autonomy and collective harmony can coexist altruistically [3].
This field is centered on the advancement of digitalsociety, and in the well-being hierarchy, our main
focus is on (2) the society/group layer, (3) individual
layer, and (4) society/group layer.

2.3 Individual
The third layer is the well-being of individuals,
whereby people with diverse values can act freely andautonomously in the pursuit of their own goals. Information and communication technology has expanded
the possibilities of individual well-being by providing individuals with a variety of choices and means oftransmitting information. Well-being in this layer isdiverse and subjective.

3.2 Lifelong health support
With the aim of creating a society where people canspend their entire lives in good health, which is themost basic part of well-being, we are studying how tovisualize individual daily data on basic lifestyles(diet, exercise, sleep) so that users can self-adjust
their mind/body rhythms in their own way [4].
This field is centered on digital representations of
the human body and exists in the life/body layer ofthe well-being hierarchy.

2.4 Self-transcendence
In his later years, Maslow claimed that there is aself-transcendence layer after the layer of self-actualization that he had previously stated to be the final
layer in the hierarchy of needs. The previous threelayers basically relate to individual well-being, but
this last layer transcends the self and relates to wellbeing gained by thinking of and acting for the benefitof others apart from oneself (which may include not
only people but also things such as specific natural
locations and the broader Earth). In a digital society,
it is said that centralized platforms can create disparities and divisions by controlling information to suit
their own business interests. To go beyond this, it
might be worth incorporating a philosophy that values self-transcendence into the governance of platforms.
3. NTT laboratories’ approach to well-being
The four approaches to well-being at NTT laboratories introduced in the Feature Articles in this issue
are explained in terms of the direction they aim totake and their position in the hierarchy of well-being(Fig. 3(b)).
3.1 Social well-being
In our view, social connections constitute the factor
with the greatest impact on well-being. However,
especially in Japan, doing well within a group of
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

3.3 Human digital twins
We are studying human digital twin technology that
reproduces not only people’s external aspects but also
their internal aspects, such as their values andthoughts [5]. From an interdisciplinary perspective,
we are looking at how this technology can contributeto well-being in the future and not to a dystopia where
humans are threatened by artificial intelligence androbots that imitate humans. We are also researching
sensory communication that allows people to convey
their feelings in digital space.
This field is centered on the digital representationof humans, and in the well-being hierarchy, our main
focus is on (3) the individual, (2) society/group, and(4) self-transcendence layers.
3.4 Human information science
Human well-being is diverse and shaped by manyfactors, including a person’s physical and mental conditions, values, and relationships with others. We aim
to obtain a comprehensive grasp of these diverse
aspects of well-being, understand the human information-processing mechanisms behind it, and study
intervention methods that operate on these mechanisms [6].
This field is centered on basic research of humans
and positioned as an area that comprehensively supports each layer of the well-being hierarchy.

20

Feature Articles

4. Future outlook

J. Watanabe and D. Chen (compilers and editors), H. Ando, K.

Sakakura, and A. Murata (editors), “Towards Collective Wellbeing—
Its Concept, Practice, and Technology,” BNN, Inc., 2020 (in Japanese).
Y. Hiroi, “Why Happiness Has Become a Social Theme Again,” Toyo
Keizai Online, 2021 (in Japanese). https://toyokeizai.net/
articles/-/455642
M. Miyamoto, Y. Nakajima, Y. Koga, Y. Nishikawa, A. Miyajima, Y.
Orime, and S. Konno, “Social Well-being: Connections that Allow
Altruistic Coexistence of Individual Autonomy and Collective Harmony,” NTT Technical Review, Vol. 20, No. 10, pp. 22–27, Oct. 2022.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202210fa2.html
H. Nakashima, M. Seyama, T. Tajima, and K. Eguchi, “Healthcare
Science for Lifelong Health—Visualizing Biological Rhythms andRegulating Them on One’s Own,” NTT Technical Review, Vol. 20,
No. 10, pp. 28–32, Oct. 2022.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202210fa3.html
A. Fukayama, S. Eitoku, I. Toshima, and S. Ozawa, “Human Digital
Twins for Well-being,” NTT Technical Review, Vol. 20, No. 10, pp.
33–36, Oct. 2022.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202210fa4.html
N. Saijo, M. Fujino, A. Murata, Y. Ooishi, and J. Watanabe, “Human
Information Science Research for Understanding and Improving Our
Well-being,” NTT Technical Review, Vol. 20, No. 10, pp. 37–43, Oct.
2022.
h t t p s : / / n t t - r ev i ew. j p / a r c h ive / n t t t e c h n i c a l . p h p ? c o n t e n t s =ntr202210fa5.html

Katsuhiko Suzuki

Masaru MiyamotoEver since the Stone Age, humankind has developed by creating tools to make life more convenient.
However, it is important to note that the tools wemake for our individual needs may, in turn, define andshape our lives. Information and communication
technology provides us with important tools that are
indispensable in today’s world, and since it has
become an indispensable infrastructure for daily life,
it has a significant impact on our well-being, for better or for worse. We will keep this in mind as we
continue our research on well-being, and by conducting research and development and outputting results
geared toward delivering a well-being society, we
will promote imaginative solutions that actively
improve people’s well-being rather than simply trying to eliminate things that make people unhappy.
References[1]

Vice President, Head of NTT Social Informatics Laboratories.
He received a B.E. and M.E. in computer science from Chiba University in 1993 and 1995.
He joined NTT in 1995 and engaged in the development of a smart card operating system. He is
now engaged in the management of the researchand development for social transformation anddevelopment by combining information scienceand social science. He is a member of the Information Processing Society of Japan (IPSJ).

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

[2]
[3]

[4]

[5]

[6]

Executive Research Engineer, Well-being
Research Project, NTT Social Informatics Laboratories.
He received a B.E. and M.E. in industrial engineering from Waseda University, Tokyo, in 1995
and 1997. He joined NTT in 1997 and conducted
research on user experience and data science bycollaborating with various partners for open
innovation. His current interest is investigating
how telecommunication companies can contribute to a future well-being society.

21Feature Articles: Research and Development toward a
Well-being Society

Social Well-being: Connections that
Allow Altruistic Coexistence ofIndividual Autonomy and Collective
Harmony
Masaru Miyamoto, Yoshiaki Nakajima, Yuzo Koga,
Yoshiki Nishikawa, Asami Miyajima, Yoshinori Orime,and Shunichi Konno
Abstract

This article describes the concept of social well-being, which seeks to facilitate the altruistic coexistence of individual autonomy and collective harmony in a decentralized society that integrates the digitaland real worlds. The article also introduces four initiatives that contribute to the accumulation of knowledge in the humanities and social sciences and a technical platform to promote behavioral change.
Keywords: well-being, decentralized society, design

1. Social well-being research overview
1.1 Problem recognition
In a recent international comparative study of wellbeing in the workplace [1], Japan ranked 5th out of
116 countries in terms of the number of people who
responded that their work significantly improves the
lives of others but came very low, 95th, in terms of thenumber of people who responded that they enjoyedtheir jobs. These results suggest that the Japanese
tend to sacrifice their happiness for the benefit of thegroup (e.g., society or the workplace). In other words,
there is a dichotomy between the interests of the individual and those of the group.
Although the digitalization of society, such as the
rapid spread of teleworking caused by the novel coronavirus (COVID-19) pandemic, presents new choices
to individuals, it can also increase the above dichotomy due to the effect of filter bubbles*1 or echo chambers.*2
As the social environment undergoes drastic changes such as population decline due to falling birthrates
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

and aging society, people are being expected to
become more mobile in their employment, andwomen and the elderly are being expected to play agreater role in society. However, employment andpromotion practices have hardly changed, and thefixed nature of roles and relationships in groups isstarting to reach its limit. This is true not only in theworkplace but also in schools and families.
1.2 Goal of initiatives
We define social well-being as connections thatallow altruistic coexistence of individual autonomy
and collective harmony and aim for a society that
*1 Filter bubble: A situation in which someone only hears or sees
news and information that supports what they already believe andlike, especially a situation created on the Internet as a result ofalgorithms (= sets of rules) that choose the results of someone’s
searches (https://dictionary.cambridge.org/ja/dictionary/english/
filter-bubble).
*2 Echo chamber: An environment in which somebody encounters
only opinions and beliefs similar to their own and does not haveto consider alternatives (https://www.oxfordlearnersdictionaries.
com/definition/english/echo-chamber).

22

Feature Articles

Decentralized society of digital/real-world fusion
Self-as-We
We = self
Altruistic
coexistenceCollective harmony

Individual autonomy
Enabling
Social well-being network

Design framework:
Design methodology,
well-being measurement

Behavior-change support technology:
Support for setting goals/planning,
lifestyle-habit improvement

Digital social rules:
Improvement of social acceptance

Environment-construction technology
for autonomous support of activities:
Support for digital/real interlocking

Humanities/social science knowledge

Technical platform

Fig. 1. Social well-being research overview.

enables social well-being. To overcome the dichotomy between individual and groups, it is importantthat people (individuals) take care of themselves
while also being considerate toward and caring forthe people and things around them.
1.3 Overview of initiatives
To facilitate such a society, we are developing asocial well-being network that consists of a technicalplatform to promote behavioral change by supporting
behavior in the digital and real worlds and the accumulation of knowledge in the humanities and social
sciences through practical designs on the basis oftheories such as philosophy and sociology. This network is introduced below (Fig. 1).
2. Initiatives that contribute to the accumulationof knowledge in the humanities andsocial sciences
2.1 A design framework for co-creating a wellbeing society
To achieve a social well-being network, we areresearching a well-being design framework. This
framework is a design methodology for co-creating
services and social mechanisms on the basis of wellbeing as a new means of measuring value as areplacement for economic growth while sharing values with diverse partners. On the basis of the knowlNTT Technical Review

Vol. 20 No. 10 Oct. 2022

edge from the humanities and social sciences, especially the concept of Self-as-We [2], we are promoting not only theoretical research but also practicalresearch on the design of a social well-being society
conducted in daily life. We aim to contribute to theactualization of a social well-being society by establishing a well-being design framework and increasing
the availability of opportunities and human resources(Fig. 2).
2.1.1 Theoretical research based on the Self-asWe concept
Based on the holistic Self-as-We concept espousing
traditional East Asian ideas as advocated by Kyoto
University philosopher Yasuo Deguchi, we are conducting research into the definition, standardization,and measurement of social well-being focused on
individual characteristics and the quality of relationships between people in groups (fellowship). We arealso researching information components and transmission methods for enabling human-to-human connections in digital and real spaces that take the Selfas-We concept into account.
2.1.2 Research on the design of a social well-being
society
By focusing on the three aspects of well-being at
work (well-working), in the classroom (well-learning), and at home and in local communities (wellbonding), we are working to create new social
mechanisms that are free of existing constraints by
23

Feature Articles

Systematizationof theory

Theoretical research(1) Theoretical research of social well-being• Research on definitions, indicators, andmeasurement methods(2) Research of Self-as-We• Research of individual characteristics,
team status, and quality of relationships
among people(3) Research of social well-being networks• Research of information elements andinformation transmission methods that
contribute to Self-as-We, assuming
digital and real-world infrastructure

Systematizationof practicalknowledge

Creating social value through co-creation• Creation of services and social
mechanisms that contribute to well-being
through co-creation conducted in daily
life with residents, local governments,and local businesses• Acquisition of practical knowledge forcreating social value

Well-being design
framework(creating well-being
services)

Utilizationof practicalknowledge

Design research

Practice

TargetWell-working/learning/bonding

Horizontal development of a social-value/well-being design framework

Fig. 2. Design-framework research overview.

engaging in co-creation with partners such as residents, local governments, and local businesses. We
are also working on the practical development oftools and processes for building relationships withpartners and generating ideas.
2.2 Digital social rules that support well-being
The growth of remote working caused by the
COVID-19 pandemic has changed the stereotypicalvalue of working at the office. The value of work-inlife is now widely recognized, and the technology
involved in web-conferencing systems makes thisvalue realistic. As described above, various technologies and services that meet the environmental andsituational requirements of society are delivering newvalues and creating a complex society where people
can hold diverse values. However, the rapid spread of
new technologies and services can create new problems. For example, the lack of company rules that
apply to remote working has given rise to a certain
amount of confusion.
However, it is difficult to anticipate issues thatmight arise when the diverse values of a society andits citizens are updated and to formulate a complete
set of rules to address these issues. Therefore, to keep
up with changes in the social environment and theemergence of new technologies and services, the ability to repeatedly update these rules within a shorttimescale while rapidly forming consensus among
various stakeholders will be necessary for ensuring
the well-being of the forthcoming digital society andits citizens, which is why we are studying how to
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

build consensus for the utilization of data in smart
cities.
Smart cities host residents and visitors with diverse
values. By acquiring, storing, and analyzing datafrom various sensing means, it should be possible toprovide new value and facilitate better urban development. However, this could give rise to issues withprivacy and data rights that are not anticipated byconventional laws and guidelines. Assuming thatconflicts may occur among the constituent stakeholders of cities, we are researching how cities, companies, and people with diverse values should cooperate
and agree on rules for data use to preserve the respective well-being of groups and individuals.
3. Initiatives that contribute to atechnical platform
3.1 T
 echnology to support intrinsic behavior
change to contribute to sustainable wellbeing
To provide long-term maintenance and improvephysical and mental health, which is a basic conditionof well-being, we are conducting research to support
behavioral goal setting and the continuation of behaviors that lead to desirable health habits for each person (Fig. 3). By observing people’s behaviors andinteractions to understand the individual characteristics and conditions that make up each person’s personality and carrying out interventions that are tailored for each individual, we are researching meansof promoting behavior change through intrinsic
24

Feature Articles

For achieving sustainable well-being by supporting the accumulation of short-term well-behavior

Well-beingA

Behavior-change support technology
based on narrative: Set agreeable
goals and action plans
Dialogue that brings out theperson’s characteristics

Narrative

Trial
and
error

B

Lifestyle-habit improvement support
technology: Planning and intervene
backwards from the goals

Life logs showing action and behavior

Personal
characteristics
Suitableintervention

Modeling a dialogue
process of experts

AI dynamicallyformulatesinterventions basedon the situation(AI: artificial intelligence)

Individual

Goals and action plansappropriate for the individual

Pre-contemplation

Life logs

Trial
and
error

Contemplation

Preparation

ActionNow isn’t
the time.

I’m planning...

How do I?
When do I?

Modeling
Suitableintervention

Stages of behavior change*

Intervention

Motivated to continue
and form a habit

MaintenanceA
No need!

time
Observation

I can continue
with this.

B* Prochaska et al, 1997

Fig. 3. Overview of behavior-change support technology.

motivation.
3.1.1 Behavior change research based on narrative: Setting goals
With a focus on human narratives, we are modelingindividual understanding and intervention methods topromote goal/action planning tasks that are currently
carried out by experts through, for example, interviews, and our aim is to systematize some of thesemodels. We are currently developing models of theunderstanding and intervention processes by analyzing actual health-guidance interviews and subjecting
them to expert reviews. As a model of a person-toperson persuasion strategy that motivates people
through interventions aligned to their inner values,
we plan to verify the effectiveness of this approach in
the area of health guidance then consider applying it
to other areas.
3.1.2 Lifestyle-habit improvement support technology: Planning backwards from the goals
We aim to support the improvement of individual
lifestyle habits by modeling behavioral patterns fromlife logs and providing adaptive action plans toachieve goals. We are currently verifying the effects
of interventions for use cases that support the
improvement of sleep and exercise habits. We will
also consider the further development of applications
that help achieve a society where people have selfNTT Technical Review

Vol. 20 No. 10 Oct. 2022

control over their habits and can take on the challengeof self-realization.
3.2 E
 ASE: Environmental configuration technology that autonomously supports people’s
diverse activities
As a technical platform for the implementation ofsocial well-being networks, we are researching atechnology called Enhanced Autonomous Supportive
Environment (EASE) that enables anyone to incorporate the benefits of digital technology in their reality
by supporting activities that link digital and real
information and functions (Fig. 4).
Although each person is an independent self (individual), they also belong to various groups such ascompanies, schools, families, and social circles.
However, when remote working came into widespread use after the outbreak of COVID-19, people
ended up having to work at home, making traditional
forms of communication more difficult. For example,
there are more questions that people have to consider
when communicating with others, such as whether
this is the right time to talk to the other person, andwhat means should be used to contact them. This is
one of the limitations of current digital technology,
and one of the issues that EASE aims to address.
EASE will bring diverse digital technologies closer to25

Feature Articles(5) Accumulate knowledge about each active
environment and people’s activities(6) Generate and distribute new skill scenarios based on knowledge

SkillsSkillsActivity-environment
intelligence

Knowledge

Skills(4) Activities performed while continuing
to interact with the environment

Knowledge

EASE

Knowledge

EASE

Front-end (PC, smartphone, wearable device, etc.)(1) Actions or situation changes triggered bya person (moving, starting a conversation,using a terminal, etc.)(3) Provide feedback by executing and linking
environmental skills in digital space andreal space

EASE(2) Invoke optimal environmental
skills by understanding the user’s
intentions and circumstances

Environmental
skillsEnvironmental
skillsService

Service

Function execution/
information acquisition
Growth potential
The more you use it, thebetter the support for
the environment.

Digital environmental-skills concierge

Digital space

LinkedEnvironmental
skillsEnvironmental
skillsDeviceDeviceCameras, microphones,
physical sensors, etc.

Information systems, cloudservices, etc.

Real space

Scalability
Environment-support
functions can be freely
added or modified.

PC: personal computer

Fig. 4. EASE, an infrastructure for supporting people.

people, enabling them to communicate smoothly
across digital and real spaces, even when the otherperson is not physically nearby.
EASE consists of three components: (i) environmental skills, (ii) digital environmental-skills concierge, and (iii) activity-environment intelligence.
Environmental skills are a set of functions that support people in using various services and devices
across digital and real spaces. The digital environmental-skills concierge makes it possible to not only
properly deploy environmental skills to actually support people but also receive feedback from people toimprove their support capabilities. Activity-environment intelligence shares information about people inremote locations, such as their knowledge and circumstances, making it possible to provide optimal
support for both individuals and groups.
By linking these components, we can, for example,
display the presence information of people who are

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

far away, autonomously operate nearby cameras andmicrophones, and control them in consideration of
the user’s relationship to the other person (colleague,
family member, friend, etc.).
4. Future outlook
We will first focus on work-in-life and regional
revitalization and co-create new knowledge, technologies, and business co-creation models throughthe creation of social values and collaborate withexperts in the humanities and social sciences.
References[1][2]

Press release issued by PERSOL Group, “‘Work, and Smile’ Global
Survey Results Preliminary Announcement,” Mar. 31, 2022.
https://www.persol-group.co.jp/en/news/20220331_370/
Natural Society Lab (in Japanese), https://group.ntt/jp/nsl/deepdive-0/26

Feature Articles

Masaru Miyamoto

Executive Research Engineer, Well-being
Research Project, NTT Social Informatics Laboratories.
He received a B.E. and M.E. in industrial engineering from Waseda University, Tokyo, in 1995
and 1997. He joined NTT in 1997 and conducted
research on user experience and data science bycollaborating with various partners for open
innovation. His current interest is investigating
how telecommunications companies can contribute to a future well-being society.

Yoshiaki Nakajima

Executive Research Engineer, Social Innovation Research Project, NTT Social Informatics
Laboratories.
He received a B.S. in information science and
M.S. in mathematical and computing sciencefrom Tokyo Institute of Technology in 1995 and1997. He joined NTT Information and Communication Systems Laboratories in 1997. He has
been involved in research and development of
information and communication platforms, security platforms, and other areas. He has certifications of Certified Information Systems Security
Professional (CISSP), Certified Cloud Security
Professional (CCSP), and Registered Information Security Specialist (RISS).

Yuzo Koga

Director, Head of Social Informatics Research
Project, NTT Social Informatics Laboratories.
He received a B.E., M.E., and Ph.D. in instrumentation engineering from Keio University,
Kanagawa, in 1996, 1998, and 2001. He joined
NTT in 2001, and has had several years of experience in designing, building, and operating various communication services including identity
management, cloud computing, the Internet
exchange point, and the Internet access network.
He is a member of the Institute of Electronics,
Information and Communication Engineers(IEICE).

Asami Miyajima

Senior Research Engineer, Supervisor, Wellbeing Design Group, Well-being Research Project, NTT Social Informatics Laboratories.
She received a B.E. and M.E. in science andtechnology from Keio University, Kanagawa, in1998 and 2000. She joined NTT in 2000, where
she engaged in the development of medical collaboration systems and security research on
cyber-physical systems. Since 2001, she has been
researching behavior-change support.

Yoshinori Orime

Senior Research Engineer, Social System
Design Group, Social Informatics Research Project, NTT Social Informatics Laboratories.
He received a B.E. in computer science from
Saitama University in 1998 and joined NTT
Network Service Systems Laboratories the same
year. He was involved with network service
development and the introduction of cybersecurity measures. Since 2020, he has been engaged
in research that incorporates legal matters andsocial requirements into technical requirements.

Shunichi Konno

Senior Research Engineer, Supervisor, Social
Implementation Promotion Group and Group
Leader, NTT-CERT Group, Social Innovation
Research Project, NTT Social Informatics Laboratories.
He received a Master’s degree in information
science and technology from the University ofTokyo in 2003. He joined NTT laboratories in2003 and investigated operating system security,
CSIRT (computer security incident responseteam) operation, and virtualization technologies.

Yoshiki Nishikawa

Senior Research Engineer, Supervisor, Living
Lab Promotion Group, Well-being Research
Project, NTT Social Informatics Laboratories.
He received a B.E. and M.E. in computer science from Waseda University, Tokyo, in 1996
and 1998. He joined NTT in 1998 and conducted
research on multimedia communication systems
and co-creation methods. His current researchinterests include human computer interaction andsocial design for a well-being society.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

27

Feature Articles: Research and Development toward a
Well-being Society

Healthcare Science for Lifelong
Health—Visualizing Biological
Rhythms and Regulating Themon One’s Own
Hiroshi Nakashima, Michiko Seyama, Takuro Tajima,
and Kana Eguchi
Abstract

The COVID-19 pandemic has brought dramatic changes to our daily lives and social activities. Anxiety over one and one’s family becoming infected, stress caused by limitations imposed on personal
behavior, changes in lifestyle, etc. have greatly affected everyone’s mental and physical condition. This
article introduces health science that aims for a society of lifelong health by visualizing one’s daily data
covering basic lifestyle habits (eating, exercising, and sleeping) and self-regulating one’s biologicalrhythms.
Keywords: healthcare science, biosensing, biological rhythm

1. Supporting a society of lifelong health
Physical well-being is one basic element of wellbeing. It is thought that good maintenance of one’s
physical and mental condition and keeping healthy isimportant and that good health can act as a basis forcultivating values (symbiosis, empathy, sustainability, happiness, and affluence) that enhance one’s quality of life. However, we are experiencing a pandemic
on an unprecedented scale in the form of COVID-19.
This pandemic has significantly degraded physicaland mental well-being throughout the world.
In November 2020, NTT announced its Medicaland Health Vision [1] to support lifelong health by
predicting a person’s physical and mental conditions
and acquiring technology to support well-being. As
part of this vision, NTT is developing bio-digital twin
technology that can sense a massive amount ofdiverse biological data in a person’s daily life, model
human physiological functions, and execute simulations for predicting the future by taking individual
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

features into account [2]. The medical field is currently undergoing a paradigm shift from symptomatictherapy that searches for a cure after a person
becomes ill to a world of care to prevent an illness
from occurring in the first place (preventive medicine). Against this background, NTT aims to conduct
digital mapping of a person’s biological data and use
advanced biological-information-processing technology to create new value, such as avoiding the risk of
disease beforehand, inducing healthy behavior tonaturally prevent illness, and supporting an individually appropriate self-reliant life.
Amid social changes, such as the trend toward
remote working, there have been reports of increasing
burden on people’s physical and mental conditions
due to working in an environment other than one’s
workplace. There are concerns that a decrease inphysical activity or irregular sleep/awake times canlead to the onset or aggravation of lifestyle-relateddiseases. To maintain one’s health, it is common to
demand endurance and tenacity to some extent in
28

Feature Articles

managing one’s behavior and establishing regular
lifestyle habits. We have therefore entered into joint
research with Waseda University on the three topics
described below in the field of healthcare science
with the aim of searching for a behavior-modification
protocol for visualizing biological rhythms andenabling automatic regulation without placing a burden on the individual. We are developing techniques
for clarifying an individual’s biorhythms by arranging lifestyle habits (eating, exercising, and sleeping)—the basic elements of well-being—along a timeaxis, accumulating biological sensing data accompanying that person’s daily behavior, and analyzing
those features. We will combine these techniqueswith knowledge related to behavioral psychology andbehavioral economics to establish a mechanism leading to automatic regulation of biological rhythms.
This could lead to the development of a technique forpresenting and indexing evidence and providing feedback so that a person’s health-related behavior can be
modified in a voluntarily, effortless manner.
2. Using circadian rhythm to improve dietary
habits (in search of chrono-nutrition)
Core body temperature (CBT) has been attracting
attention as one index reflecting the circadian rhythmof a person’s body. As shown in Fig. 1, CBT refers to
the temperature of the body’s core including the brain
and other organs. To keep these organs functioning,
CBT is not easily affected by the outside environment
and is the highest temperature in the body. CBT rises
as a result of an inflammatory response, such as a heatstroke or infectious disease, and drops at the onset ofa condition such as hypothermia; thus, it is used as animportant vital sign in medicine. CBT fluctuatesslightly by about 1°C within the range of normal
body temperature in a daily cycle. This fluctuation islinked to the circadian rhythm of every individual [3].
To visualize circadian rhythm based on CBT, NTT
has been researching and developing a high-accuracy
wearable CBT sensor that can detect this slight daily
fluctuation in CBT without placing an excessive loadon the body. Medical settings, such as surgery, use
measurement methods that involve directly inserting
probes into organs, such as the rectum or ear (eardrum temperature), but these methods can place aheavy burden on the patient. In contrast, NTT is using
a mechanism that estimates CBT by measuring theflow of heat radiated from the body core, as shown in
Fig. 1, to the skin using a sensor attached to the surface of the body [3]. The way in which heat escapes
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

CBT

Body core

Peripheral bodytemperature

Fig. 1. Peripheral body temperature and CBT.

from the body differs in accordance with the outsideenvironment such as wind, temperature, and humanactivities, but the CBT sensor under development at
NTT is equipped with a function for suppressing theeffects of fluctuations in the outside environment
when the person is in a restful state.
The biological clock that generates the human circadian rhythm is regulated by the input of information such as external light on a specific brain area
(suprachiasmatic nucleus). Recent research has
revealed that this biological clock affects the peripheral clocks of various organs such as the liver, kidneys, and muscles, which indicates that the circadianrhythm has a close connection with various states ofour body such as metabolism, quality of sleep, exercise, and onset of illness. For example, the level of
glucose in the blood when eating can easily increasein the evening hours compared with the afternoon asa circadian rhythm [4]. It is therefore expected that anindividual can achieve a healthier dietary lifestyle byingesting an appropriate meal at the appropriate time
given an understanding one’s circadian rhythm (chrono-nutrition). In other words, an individual should be
able to acquire healthy habits without excessive effort
by incorporating nutrition that takes into account thisbiological clock in one’s daily life. Chrono-nutrition
has been tested mainly in nocturnal animals such as
mice. In our joint research with Waseda University,
we are applying NTT’s CBT sensor to chrono-nutrition and gathering evidence on the correlation
between the diurnal circadian rhythm of humans anda healthy diet.

29

Feature Articles

Normalcircadian
clock
Sleep

Activities

CBT

Rest

Circadian rhythmwith shift

Circadian rhythmwith no shift
0

8

20

Living time (hour)Fig. 2. Circadian rhythm visualized using CBT.

Environment control such as lighting,air conditioning, and diet

Data analysis

CBT measurement

NTT’s CBT sensor
under development

Fig. 3. Example of a healthcare system using CBT measurement.

3. Visualizing circadian rhythm to treat
sleep disorders
In today’s society in which people’s lifestyle habits
are becoming increasingly diversified, a shift can easily occur between the circadian rhythm and living
time, such as the time that one goes to bed and wakes
up, and that this shift is related to the quality of sleep.
With no shift between the circadian rhythm and living
time, CBT begins to drop several hours before sleep
then starts to increase in the latter half of sleep, asshown in Fig. 2. However, when a shift does occur
between the circadian rhythm and sleep/awake times
within one’s living time due to an irregular lifestyle,the quality of sleep deteriorates. This state is calledsocial jet lag, which is similar to ordinary jet lag tosome extent. If left unchecked, this state can lead tosleep disorders such as difficulty falling asleep, lightsleep, difficulty in getting out of bed, and sleepiness
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

during the day, which can have a negative effect on
one’s mental and physical health and social activities[5]. To enable an individual to easily regulate andimprove one’s quality of sleep, it is important toacquire a technique for one to effortlessly carry out
sleep self-management while understanding theircircadian rhythm. Applying such a technique shouldmake it possible to construct a healthcare system oriented to the individual, as shown in Fig. 3.
We are conducting pilot research on how knowledge of the circadian rhythm obtained from NTT’s
CBT sensor can help improve sleep quality. We areconducting long-term CBT measurements regarding
daily life, and in addition to evaluating biologicalrhythms obtained from data on the diurnal variationof body temperature, we are conducting correlation
analyses with data obtained from a sleep-evaluation
wristband, stress-marker tests (cortisol and amylasein saliva), and stress-check questionnaires. Our aim is30

Feature Articles

to closely track any correlations between temperaturepatterns obtained from the CBT sensor and stressmarker concentrations or stress indicators, which wehope will lead to methods for evaluating sleep quality
and guidelines for improving life.
4. Testing effects of online health and hygiene
guidelines on remote workers
Remote working has been expanding since 2020 in
the midst of the COVID-19 pandemic. However, adecrease in daytime activities, insufficient exercise,and disruption in one’s sleep/awake rhythm due to alife centered on the home can be associated withlifestyle-related diseases and poor mental health. As
reflected by comments such as “I can sleep well ondays that I move around a lot” and “I feel well-rested
with no fatigue after a good night’s sleep,” nocturnal
sleep and daytime exercise are closely related lifestyle habits. Consequently, in this new normal era, itis thought that each person should change their attitude and behavior toward sleep and exercise and thatthe need for ongoing healthy behavior will increase.
Against this social background, we have begun to
study awareness-raising and behavior modification
centered on sleep and exercise. As the first step of thisstudy, we are conducting online health and hygiene
education seminars targeting remote workers in collaboration with a board certified physician of theJapanese Society of Sleep Research and analyzing
their effects. We are using subjective evaluations
related to sleep, exercise, and work conditions (commuting to the office or working at home) obtained
from questionnaires as well as objective indicesrelated to personal activity (number of steps, calories

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

burned) and body composition (weight, body massindex) obtained from wearable devices or othermeans. Our aim is to gain a multifaceted understanding of changes in the daily behavior of remote workers. We are also attempting to clarify the changes indaily behavior induced by online health and hygiene
education seminars and the relationship between daytime activities and nocturnal sleep. We plan to use theknowledge gained through these studies to investigate techniques for increasing awareness and changing daily behavior for the better in an effortless andsustainable manner. We will also strive to explain any
ripple effects of changes in daytime activities andnocturnal sleep in preventing lifestyle-related diseases.
References
[1]

[2]

[3]

[4]

[5]

Press release issued by NTT, “Medical and Health Vision: Realizationof a Bio Digital Twin—NTT Corporation Working on the ‘Future of
Medicine’ to Predict Diseases, and Mental and Physical Conditions,
Enabling People to Have Health and Hope for the Future,” Nov. 17,2020.
https://group.ntt/en/newsrelease/2020/11/17/201117c.html
H. Nakashima, K. Hayashi, and H. Gotoh, “Data-driven Medical and
Health Support Created Using Bio-digital Twin,” NTT Technical
Review, Vol. 19, No. 7. pp. 17–22, July 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202107fa1.html
D. Matsunaga, Y. Tanaka, T. Tajima, and M. Seyama, “Technology for
Visualizing the Circadian Rhythm: Wearable Core-body-temperature
Sensor,” NTT Technical Review, Vol. 19, No. 7, pp. 34–39, July 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202107fa4.html
K. Kitamura, X. Zhu, W. Chen, and T. Nemoto, “Development of a
New Method for the Noninvasive Measurement of Deep Body Temperature without a Heater,” Med. Eng. Phys., Vol. 32, No. 1, pp. 1–6,
Jan. 2010.
S. Hashimoto, S. Honma, and K. Honma, “Sleep and Biological
Rhythms,” Folia Pharmacological Japonica, Vol. 129, No. 6, pp.
400–403, June 2007 (in Japanese).

31

Feature Articles

Hiroshi Nakashima

Executive Research Scientist and Project Manager of Bio-Medical Informatics Research Center, NTT Basic Research Laboratories.
He received a B.E., M.E., and Ph.D. in appliedchemistry from Waseda University, Tokyo, in
1995, 1997, and 2002. He joined NTT Basic
Research Laboratories in 1997. Since then, he
has been engaged in research on the synthesis
and control of the optoelectrical properties of(semi)conductive polymers and nano-bio materials. His recent work includes research on biomonitoring using the functional material hitoeTM
wearable bioelectrodes. He was a visiting
researcher at University of Oxford, UK in 2002
and at University of Wollongong, Australia in
2010–2011. He is a member of the Chemical
Society of Japan (and its Division of Colloid andSurface Chemistry) and the Japan Society ofApplied Physics (and its Division of MolecularElectronics and Bioelectronics).

Michiko Seyama

Senior Research Engineer, Supervisor, NTT
Device Technology Laboratories and Bio-Medical Informatics Research Center, NTT Basic
Research Laboratories.
She received a B.S., M.E., and Ph.D. in appliedchemistry from Waseda University, Tokyo, in
1995, 1997, and 2004. In 1997, she joined NTT
where she worked on an odor-sensing system
based on a plasma-polymerized organic film forenvironmental contamination and biogas, and a
1-D surface plasmon resonance biosensor combined with a microfluidic device for on-site
immunological sensing. Her current interests
include development of biosensing platforms forblood components and blood coagulation,
including invasive and noninvasive technologies.
She is a member of the Japan Society of Applied
Physics and the American Chemical Society.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Takuro Tajima

Senior Research Engineer, NTT Device Technology Laboratories and Bio-Medical Informatics Research Center, NTT Basic Research Laboratories.
He received a B.S. and M.E. from the University of Tokyo in 2000 and 2002, and Ph.D. from
Tokyo Institute of Technology in 2017. Sincejoining NTT in 2002, he has been engaged in the
research and development of laser photoacoustic
spectroscopy and broadband dielectric spectroscopy for non-invasive biomedical sensing. His
current research involves multi-modal biomedical analysis using microwaves, terahertz-waves,
optics, and ultrasonics. He is a member of theInstitute of Electronics, Information, and Communication Engineers (IEICE) of Japan.

Kana Eguchi

Research Engineer, NTT Communication Science Laboratories and Bio-Medical Informatics
Research Center, NTT Basic Research Laboratories.
She received a B.E. from the Kyoto Institute ofTechnology in 2010, and M.S. and Ph.D. in informatics from Kyoto University in 2012 and 2020.
She joined NTT in 2012, and her current researchinterests include biosignal processing, wearable/ubiquitous computing, medical engineering, andmedical informatics. She is a member of IEICE,
the IEEE (Institute of Electrical and Electronics
Engineers) Engineering in Medicine and Biology
Society, the Japanese Society for Medical andBiological Engineering, and the Japanese Society of Sleep Research.

32

Feature Articles: Research and Development toward a
Well-being Society

Human Digital Twins for Well-being
Atsushi Fukayama, Shin-ichiro Eitoku, Iwaki Toshima,and Shiro Ozawa
Abstract

NTT is researching digital twin technology that reproduces not only the outer states, such as appearance, voice, and movement, but also the inner states, such as values and thoughts, of humans. To actualize a future in which human digital twin technology contributes to the well-being of people and society,
it is necessary to examine the ideal path of technology evolution from a broad interdisciplinary perspective. This article introduces research activities to pursue well-being as the Grand Challenges of Digital
Twin Computing, i.e., Another Me and Mind-to-Mind Communication.
Keywords: digital twin, well-being, Self-as-We, Another Me, Mind-to-Mind Communication

1. Toward a human digital twin for well-being
A digital twin reproduces physical entities, i.e.,
humans, objects, and the environment, on a computeras twins in the digital world on the basis of data collected about such entities. NTT is engaged in researchand development for the actualization of various
digital twins and their applications from human communication to simulation of cities and transportation[1, 2]. A human digital twin should have not only an
external model of a person’s face, body, voice, andmovement but also an internal model of a person’s
personality, values, and knowledge. When one hears
this, they may be afraid that others would look into
their inner self to exploit them or that their existence
could be replaced with their digital twin.
To eliminate the possibility of adverse effects
caused by the unethical use of human digital twin
technologies and pursue ethical advances in suchtechnologies to contribute to the well-being of people
and society, NTT has established two Grand Challenges, i.e., Another Me and Mind-to-Mind Communication [3]. A human digital twin should not be used
to substitute its original person but facilitate that
person’s connection with other people and create newopportunities through deep understanding of that person, which would expand the possibility of self-realization and achieve a well-being society based onstrong solidarity and altruistic coexistence.
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

This concept is similar to the Self-as-We theory
proposed by our joint research partner, Professor
Yasuo Deguchi of Kyoto University. In the Self-asWe theory, a “self” does not mean “I” as an entity thatthinks and therefore exists here and now but means“we” that includes all persons who share a purpose or
behavior and also objects that support it. Therefore,
the “we” that I belong to, i.e., the “self,” will change
in various ways depending on what goals I am working toward and what activities I am engaged in. A
human digital twin that deeply understands the person serves as a bridge that connects that person andothers, enabling that person to discover a new self andaim for a better “we” that is open to diversity.
The Grand Challenge Another Me is based on theconcept that the digital twin of me that precisely
reproduces the outer and inner states of me performs
various activities autonomously on behalf of me inthe same way as how I would do in those activities(for example, one’s digital twin will make friends
autonomously in cyberspace), and that by giving
feedback with a sense of reality to me, I could expand
opportunities in my life, leading to personal wellbeing such as self-realization.
Mind-to-Mind Communication contributes to asociety full of diversity where people can mutuallyrespect each other as well as a society full of creativity in which people’s ideas are combined by understanding each person and preventing miscommunication
33

Feature Articles

Comparison

Time

Time

Me

Me

AnotherMe

Another
Me

Me<Extended Indexical I>
Indexical I consists of I and Another Me
Function that I can observe
Function that I cannot observe

Fig. 1. Functional I.

(discrepancies in intentions between information
sender/recipient) and discommunication (a state inwhich information transmission itself is missing).
The following is a discussion on well-being in thecontext of Another Me and Mind-to-Mind Communication.
2. Another Me: another way of being self
Another Me is not an expansion of a uniform person through impersonal artificial intelligence (AI) but
an expansion of a person’s temporal and spatial
potential while maintaining the diversity that each
person has. However, depending on the situation inwhich it is used, there is a possibility that Another Me
would replace its actual person, thus degrade the irreplaceability and dignity of that person. What is thedesirable position of Another Me in a society thatenhances well-being? To answer this question, we areconducting joint research with Professor Yasuo
Deguchi and Associate Professor Takuro Onishi of
Kyoto University from a philosophical perspective.
After many discussions through collaboration ofphilosophy and technology, we concluded that
Another Me, as an existence having high I-hood but
not “I” itself, can be positioned on two axes. The firstaxis is Functional I (Fig. 1), which represents thesimilarity of what Another Me and “I” can and cannot
do. Functional similarity is necessary to allow Another Me to act on behalf of “I.” The other axis is Indexical I (Fig. 2), which highlights an aspect of “index”
that refers to “I” on the basis of certain evidence or aclue found in Another Me that evokes the sensation of“I’m in this now.” We discussed that indexicality canbe expanded with the Self-as-We theory. If we think
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

1) Connectedness: the feeling that could be obtained when“I” share time, space, and experiences with Another Me[Strong Connectedness = Close distance]
Me

Another Me2) Ownness: the feeling that I am informed about what Another Me
does, and vice versa, making me feel like Another Me is a part of “I”[Strong Ownness = Thick line]
Indexical I is summation of Connectedness
and Ownness in time axis.

Fig. 2. Indexical I.

“I” and Another Me as a “self” on the basis of thistheory, through the sense of “acting and experiencing
together with Another Me,” we can feel “I” in Another Me. The Indexical I is thought to consist of two
further elements. One is “Connectedness,” which isthe feeling that could be obtained when “I” sharetime, space, and experiences with Another Me, andthe other is “Ownness,” which is the feeling that I am
informed about what Another Me does, and viceversa, making me feel like Another Me is a part of “I.”To enhance the well-being of Another Me throughthe expansion of human temporal and spatial possibilities, we believe it is important to enhance both the
Functional I and Indexical I elements of Another Me
both technically and design-wise (Fig. 3).
3. Two communication challenges of
Mind-to-Mind Communication technology
Figure 4 shows two communication issues that
Mind-to-Mind Communication technology needs toovercome. Figure 4(a) shows a communication scene
in which you and your partner have different purposes, backgrounds, and sensibilities. In such a case,communication may not be established in the firstplace, and discommunication may occur. The purpose of Mind-to-Mind Communication technology isto understand the general meaning. In other words,
the goal is to convey, even if vaguely, that there is a34

Feature Articles

100

I

Indexical I (%)*Conceptual
Though it is another being, diagram
think of it as me
Though there is a difference,
think of it as a part of me
Not me and not a part of me

00

Functional I (%)

100

Fig. 3. Proportion of Functional I and Indexical I.

Different
sensibility

Doesn’t talk about the sensibility differenceOccurrence of discommunication
Support for overall understanding
Acceptance of diversity

Different background,purpose(a) Resolution of discommunication and diversity inclusion

Different
sensibility

Discrepancy from the sensibility differenceOccurrence of miscommunication
Support for understanding the difference
Mutual understanding and better consensus building

Same background,purpose(b) Resolution of miscommunication and mutual understanding

Fig. 4. Challenges of Mind-to-Mind Communication and their effect.

sensibility different from one’s own. Even if we do
not understand the details of sensibility, if we cansense and recognize that there are various sensibilities, it will promote understanding and acceptance ofdiversity. Figure 4(b) shows a scene with a common
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

background and purpose and different sensibilities.
This occurs often in everyday life. Differences insensibility lead to miscommunication and misunderstanding. However, if two people with different sensibilities can understand each other for the same
35

Feature Articles

purpose and form a consensus, it will lead to better
consensus building and, in turn, increased creativity.
As a use case to promote specific technologicaldevelopment aimed at overcoming discommunication and promoting the acceptance of diversity, we
are working to support communication for people
with mental disabilities and in minority groups in theworkplace. In particular, we are building prototypesof support systems targeting the problem that occurs
when each person has a different sensibility, erroneous assumption, and misunderstanding, which could
cause problems in the workplace and limit the person’s opportunity for active contribution. To visualize
and understand well-honed sensibility and enhancecreativity beyond differences in sensibility, we havestarted to convert sensibilities between professionals,
such as Shogi (Japanese chess) players and Formula
racing teams, and analyze the dialogue and movement between players with different expertise toimprove the performance of the individual players. In
Shogi, there are no unstable factors for movement

Atsushi Fukayama

Senior Research Engineer, Group Leader,
Another Me Research Group, NTT Digital Twin
Computing Research Center.
He received an M.S. in informatics from Kyoto
University in 1999 and joined NTT the same
year. After working on media-recognition technology, research and development of humancomputer interaction technology, and practicalapplication development of network services, hehas been leading Another Me Research Group at
the NTT Digital Twin Computing Research Center since 2021.

Shin-ichiro Eitoku

Senior Research Engineer, NTT Digital Twin
Computing Research Center.
He received an M.E. and Ph.D. in information
science and technology from the University ofTokyo in 2006 and 2013. He joined NTT in 2006.
His research interests include information systems and multimedia systems for communications.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

due to the environment, and the excellence of agreement and conclusion can be objectively evaluatedthrough recent AI. Because, in many cases, a Formula team deploys two cars with the same conditions
in each competition, two drivers with the same conditions and a well-honed sensibility interact with eachother from nearly the same environment, allowing
them to compare utterances. Although we have only
just begun, we are also working to encourage creativity by studying the sensibilities of professionals.
References
[1]

[2]
[3]

“Feature Articles: Digital Twin Computing for Advanced Interaction
between the Real World and Cyberspace,” NTT Technical Review,Vol. 18, No. 9, pp. 13–39, 2020. https://ntt-review.jp/archive/2020/202009.html
White paper on Digital Twin Computing,https://www.rd.ntt/e/dtc/DTC_Whitepaper_en_2_0_0.pdf
R. Kitahara, T. Kurahashi, T. Nishimura, I. Naito, D. Tokunaga, and
K. Mori, “Research and Development of Digital Twin Computing for
Creating a Digitalized World,” NTT Technical Review, Vol. 19, No.
12, pp. 16–22, Dec. 2021. https://doi.org/10.53829/ntr202112fa1

Iwaki Toshima

Senior Research Engineer, NTT Digital Twin
Computing Research Center.
He received an M.E. from the University ofTokyo in 2002 and joined NTT the same year. He
then received a Ph.D. in engineering from Tokyo
Institute of Technology in 2008. His researchinterests include human-computer interaction,
robot audition, and digital twins.

Shiro Ozawa

Senior Research Engineer, Supervisor, NTT
Digital Twin Computing Research Center.
He received a B.E. and M.E. from Tokyo University of Mercantile Marine (now, Tokyo University of Marine Science and Technology) in1997 and 1999. After joining NTT as a researcher in 1999, he worked for NTT Cyber Space
Laboratories (now, NTT Human Informatics
Laboratories). His fields of interest are computer
vision, video communication, user interfaces,
human-computer interaction, and 3D displays.
He is a member of the Institute of Electronics,
Information and Communication Engineers, theInstitute of Image Information and Television
Engineers, and the Institute of Image Electronics
Engineers of Japan.

36

Feature Articles: Research and Development toward a
Well-being Society

Human Information Science
Research for Understanding and
Improving Our Well-being
Naoki Saijo, Masahiro Fujino, Aiko Murata,
Yuuki Ooishi, and Junji Watanabe
Abstract

Our well-being takes on a variety of forms—it can be shaped by many factors such as an individual’s
physical and mental states, values, and even relationships with other people. We would like to contributeto improving people’s well-being by comprehensively grasping such diverse forms of well-being, understanding the human information processing mechanisms lying in the background, and devising intervention methods to act upon those mechanisms. This article introduces our ongoing research on understanding and achieving well-being based on human emotions.
Keywords: emotion, mindfulness meditation, measurement of well-being

1. People’s well-being and human information
science research
At Human Information Science Laboratory in NTT
Communication Science Laboratories, we aim todevelop information and communication technologies that enable heart-to-heart communication. Thus,
we are researching information processing mechanisms involved in perception, cognition, action, andemotion, focusing on the human brain and body fromthe three viewpoints of information science, psychology, and neuroscience. In everyday life, we feel theworld around us through our eyes, ears, nose, tongue,
and skin, i.e., our sensory organs, and move our bodies and act upon the world accordingly. Through thisinterface that we call the human body, we can engage
in heart-to-heart communication by accepting another person as he or she actually is and touching that
person’s heart. Emotions are essential in heart-toheart communication yet difficult to control as one
would like. When receiving the hearts of others, emotions include things that appear unconsciously as aphysical reaction (emotional reaction), which, inturn, includes things that one is aware of and recogNTT Technical Review

Vol. 20 No. 10 Oct. 2022

nizes (emotional cognition = feelings). These interact
with each other and greatly shape our perceptual
experiences and behavior in daily life.
These emotions are deeply involved with our wellbeing, which is a vibrant state that we can vividly
feel. They constitute a subjective experience thatincorporates an emotional reaction that arises when
we receive information from the environment or fromanother person, and within that, emotional cognition
that we can be actively conscious of. In other words,
well-being is something that is greatly affected by atransition in emotions. For this reason, it is thoughtthat understanding the mechanisms of human emotions and handling them appropriately can help
improve well-being.
2. Research into mindfulness meditation as anemotion control method for regulating physicaland mental states
Emotions that change from day to day can enrich
our lives at times and make us suffer other times.
Coping effectively with these fluctuating emotions
and being in a vibrant state is the basis of individual37

Feature Articles

Mindfulness meditation

Focused attention
meditation

Open monitoring
meditation

Narrow focus on breath

Observe various types of
information as they are

Fig. 1. Focused attention meditation and open monitoring meditation making up mindfulness meditation.

well-being. However, in modern everyday life, full ofstress brought on by a rapidly changing environment,
information overload, interaction with many people,
etc., achieving well-being is not so simple. Under
these conditions, mindfulness meditation has been
attracting attention. Today, an increasing number ofcompanies are introducing meditation as a method
for reducing employee stress and improving focus
and productivity. Some schools are also adopting it asa method for reducing stress in children and helping
them manage their feelings. This, as well as the development of many types of meditation apps, reflect the
rapid adoption of meditation in society. Against thisbackground, we are researching the physiological,
psychological, and neural mechanisms triggered by
the practice of mindfulness meditation with the aimof improving the well-being of people by establishingnew intervention methods based on scientific knowledge.
3. Physiological basis of stress-reduction effects
caused by mindfulness meditation
The awareness that arises from paying attention in
the present moment, and non-judgmentally is called
mindfulness. Mindfulness meditation, which seeks toachieve this state, is a form of mental training thatbegins by focusing one’s attention on a single target,
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

such as one’s breath, then gradually expanding tovarious senses, feelings, and thoughts. The aim is tocontrol one’s attention and emotions without being
swayed or confused by those targets of attention.
Many studies have shown that this form of meditation
can effectively reduce stress, prevent recurrences ofdepression, and improve well-being.
The physiological mechanisms behind this reduction in stress by mindfulness meditation haveremained unclear. Autonomic activity and concentrations of stress hormones have been used as indicatorsfor measuring stress, but the effects of mindfulness
meditation on these physiological indicators havebeen inconsistent. Therefore, we looked at mindfulness meditation as a combination of two meditation
methods: focused attention meditation for focusing
one’s attention on a single target and open monitoring
meditation for recognizing various senses, feelings,
and thoughts as they are [1] (Fig. 1). We hypothesizethat the effects of either one or the other meditation
method was presented or the effects of both was presented in mixed form in previous studies, which suggests that inconsistencies would likely appear among
those results.
We developed a 30-minute period of instruction forboth focused attention meditation and open monitoring meditation that would enable even people with noexperience in meditation to practice each method. We
38

*

**

2
SDNN/rMSSD

rMSSD (ms)

50
40
30

Salivary cortisol concentration(pg/ml)Feature Articles

1.5

1

20
Focusedattention
meditation

Open monitoring
meditation

Focusedattention
meditation(a) Parasympathetic
activity indicatorOpen monitoring
meditation(b) Sympathetic activity
indicator

**
1000
800
600
400
Focusedattention
meditation

Open monitoring
meditation(c) Cortisol level

Focused attention Open monitoring
meditation
meditation

Before meditating
While meditating(after meditating in cortisol)(error bar represents a 95% confidence interval)( *: p < 0.05;**: p < 0.01)

Physical activity

Physical activity
meaningParasympathetic activity

Physiologically relaxed

Sympathetic activity

Physiologically active

Rises

Cortisol levelStress hormones

Drops

Rises

rMSSD: root mean square successive difference
SDNN: standard deviation of normal and normal interval(d) Summary of physical state changes

Fig. 2. Changes in physical state caused by mindfulness meditation.

then investigated changes in participants’ physiological stress levels caused by focused attention meditation and open monitoring meditation. The results
indicate that focused attention meditation triggered arise in parasympathetic activity, the physiologicalbasis of relaxing (Fig. 2(a)). Open monitoring meditation, however, started an increase in sympathetic
activity, the physiological basis of arousal (Fig. 2(b)),
and a simultaneous drop in cortisol level, which issaid to be a stress index (Fig. 2(c)). These results canbe interpreted to mean that focused attention meditation has the potential of ridding oneself of distracting
thoughts and enabling relaxation by focusing on asingle target. During open monitoring meditation, in
contrast, people’s arousal level is high because theyexperience various senses, feelings, and thoughts,
whereas their stress is reduced because they experience them as they are (Fig. 2(d)). Therefore, it has
been shown that focused attention meditation andopen monitoring meditation activate different physiological mechanisms and contribute to stress reduction in different ways [2].
By effectively combining focused attention meditaNTT Technical Review

Vol. 20 No. 10 Oct. 2022

tion and open monitoring meditation, the possibility
arises of improving our well-being on our own by
appropriately controlling our inner state as in making
changes to autonomic activity, hormone levels, etc.
4. Measuring diverse factors in everyone’s
well-being
Our well-being takes on diverse forms—it is notdetermined solely on the basis of an individual’s
physical and emotional states. It can also change
according to what a person values and how that person understands his or her relationships with otherpeople, the environment, etc. In short, a multifaceted
approach must be taken to obtain a deeper understanding of well-being. With this in mind, we aredevising original techniques for measuring (1) changes in self-reported physical and mental states, (2)values that need to be satisfied for oneself, and (3) theway in which people perceive the relationships withothers (idea of self), and testing the validity of each ofthese techniques.

39

Feature Articles

Please select one
word that matches
your current stateof mind.

BrightExperience sampling: Multiple reports of physical and mental state in one day

BrightFrustrated

Uh-huhHmm

Daily well-being
Was today a good day?
Would you like to spend
such a day again?

(a) Procedure for recording sensitivity expressions several times a day using a mobile terminal

Bad day

Sensitivity
expressions
for a day of
low well-being

Sensitivity
expressions
for a day ofhigh well-being

うきうき (uki-uki)
わくわく (waku-waku)
いきいき (iki-iki)
にこにこ (niko-niko)
わーい (waai)
すっきり (sukkiri)
ほっこり (hokkori)
はらはら (hara-hara)
うとうと (uto-uto)
しゃきーん (shakiin)
ほっ (ho)ふう (fuu)
よし (yoshi)
ぼーっ (boo)
うんうん (un-un)
よっしゃぁ (yosshaa)
ぴりぴり (piri-piri)
わあ (waa)
へとへと (heto-heto)
どきどき (doki-doki)
おえー (oee)
そわそわ (sowa-sowa)
やれやれ (yare-yare)
がーん (gaan)
うーん (uun)
えー (ee)ぎゃあ (gyaa)
むかむか (muka-muka)
ざわざわ (zawa-zawa)
いらいら (ira-ira)
びくびく (biku-biku)
もやもや (moya-moya)
はーあ (haa)
ぐったり (guttari)
しょんぼり (shonbori)
げぇ (gee)
がっくり (gakkuri)

Daily well-being

Good daySound symbolic words(b) Relationship between daily well-being and sensitivity expressions

Fig. 3. Experience-sampling method using sensitivity expressions.

4.1 I ntuitive and embodied experience-sampling
method for monitoring physical and mental
state transitions
Emotions have aspects of physical emotional reactions and subjective emotional cognition, so to learn
about the inner state of an individual, it is necessary
to measure changes in that person’s subjective mental
state in addition to objective biological indicatorssuch as hormone levels and autonomic nerve activity.
One technique for evaluating subjective physical andmental states is the experience-sampling method.
This method is a research procedure for monitoringever-changing subjective emotions, thoughts, feelings, behaviors, etc., with respect to a particular eventat that instant or shortly after. This self-reporting is
performed several times a day, so a method thatmakes for easy and intuitive replies is desirable.
Therefore, we decided to use sound symbolic words
and interjections in Japanese, which can be considered subjective expressions with embodied intuitions
NTT Technical Review

Vol. 20 No. 10 Oct. 2022[3]. This method records one’s emotions at a certain
moment several times a day using such words, e.g.,“doki-doki,” used in a sense similar to the English“thump-thump,” and interjections such as “waai”
used in a sense similar to the English “wow,” whichmakes it possible to examine transitions in subjective
mental state with a minimal cognitive load on the user
(Fig. 3(a)).
On actually using this method to investigate therelationship between daily changes in mental state
and well-being (16 subjects × 4-week survey), andtaking as an example a day that participants thoughtof as “a really good day” (high well-being), we foundthat there were many replies such as “uki-uki ()”
indicating good expectations of the future or “iki-iki()” and “niko-niko ()” reflecting a
positive state compared with replies like “yosshaa()” indicating instantaneous joy (Fig. 3(b)).

40

Feature Articles

Survey image

Our
Well-being

Factor 1
Immersion

Factor 2
Accomplishment

Factor 3
Experience sharing

When concentrating
on something, forget
irrelevant things bythinking of nothingelse

After improving in acertain skill, feel thesignificance of one’s
efforts with regard tothat behavior

Feel a sense of togetherness when anotherperson has the same
feelings under the same
conditions as oneself

ExamplesWe

Society

Universe

Personal factors

Factors in involving
oneself with others

Factors in involving
oneself with thecommunity andsociety

Factors in involving
oneself with wider
targets (world/nature)

Cooperation; social
contributions; diversity;order

Connect with nature;peace; connect with life;
connect beyond time

Categories of psychological factors in well-beingI

WE • SOCIETY

UNIVERSE• I have an objective.

• I can share mysensitivities.
• I can be accepted byothers.
• I have friends who I canrely on.
• I and others can accept
each other.
• I place importance onfamily.
• I don’t compare myself
with others.
• I don’t read between
the lines.
• I spend only some time
together.
etc...

• The world is at peace.
• I can accept diverse
values.
• I want to contribute tosociety.
• I want to pursue theessence of the world.
• I want to see beautiful
scenery.
• I want to be one withnature.
• I hold idols and artists
in high esteem.
etc...

• I can do what I
want to do.
• I can do it because
it’s me.
• I can like myself.
• I read books andlisten to music I like.
• I immerse myself
in books and games.
etc...

Our Well-being Cards
I

Enthusiasm/immersion; Value sharing; appreciation;
challenge; achievement; yearn/respect; mutual trust
mindfulness* Only some cards shown

Source: J. Watanabe and D. Chen (compilers and editors), “Towards
Collective Wellbeing—Its Concept, Practice, and Technology,” BNN(2020), p. 17 (in Japanese).

(a) Categorization of free descriptions based on survey(b) Creation of 27 cards based on those categories.

Fig. 4. Creation of Our Well-being Cards.

4.2 Tools for visualization and sharing values
Things that are important to ones or mentally satisfy oneself differ from person to person. The firststep toward respecting and achieving diverse forms ofwell-being is to remind oneself of one’s values andnotice similarities and differences between these values and those held by surrounding people, thereby
fostering mutual understanding.
We conducted a questionnaire-based survey targeting about 1300 people asking them to write down
three things “important to you” and categorized theapproximately 3900 replies obtained (Fig. 4(a)). We
found that we could categorize those replies intothings concerning oneself (I), things concerning
involvement with other people (We), things concerning involvement with the community and society(Society), and things concerning the world, nature,and other broad subjects (Universe) [4]. We thenextracted main factors from those four categories andcreated 27 “Our Well-being Cards” (Fig. 4(b)) [5].
These cards can be used as a tool for promoting
awareness of what values one and surrounding peoplehold.
We have used these cards in a workshop targeting
elementary school students. Groups of students
shared what was important to each in terms of wellNTT Technical Review

Vol. 20 No. 10 Oct. 2022

being then devised a travel plan that could achievewell-being for all members of the group.
The students commented that “I was surprised thatsome things were the same as those of other students
while other things were different” and that “The storyof a trip can grow infinitely when other people arepresent.” We found that such an activity could promote mutual recognition of diverse values.
4.3 M easurement of holistic idea of self—Self-asWe
It has been pointed out that how one perceives oneself in relation to others, that is, one’s idea of self,
differs according to culture and the individual. In collaborative research with Professor Yasuo Deguchi, aphilosopher at Kyoto University, we developed the
Self-as-We scale to quantitatively evaluate the Selfas-We concept proposed by Professor Deguchi on thebasis of a technique in psychology for measuringpersonal characteristics [6].
The conventional idea of self is that “I” myself canexist in an autonomous manner since I can manipulate tools and control my environment even without
having to interact with other people. However, Selfas-We views “we” consisting of everything such asother people and the environment as “self” and41Feature Articles

Self-as-We“We” = self

Conventional self
“I” = self

“We” = selfI
self

Control

Entrust

I
Body

Other
peopleBody Other people• I have an autonomous existence and can survive
even without involving myself with other people.

• I live within “we” and cannot do anything on
my own.

• I feel that I can freely control my own body andinvolvement with other people.

• I treat “we” as self and I and my body and others
as being entrusted by “we.”Fig. 5. Self-as-We. Concept of conventional self (left) and Self-as-We (right).

considers “I,” as a member of “we,” to be entrusted by“we” for some of the same actions as other people
and things (Fig. 5). In other words, people and things
within “we” can be viewed as equals deserving ofrespect. The Self-as-We scale quantifies such involvement with the environment and other people througha multi-item questionnaire.
On surveying the relationship between mental
health and the idea of self during the COVID-19 pandemic using the Self-as-We scale, we found thatpeople who were more strongly inclined to have a
Self-as-We view were less likely to be depressed, thatis, they were in better mental health [7].

phy, and design.
References
[1]

[2]

[3]

5. Conclusion[4]

To contribute to improving our well-being, it isnecessary to understand the underlying mechanismsof human emotions and understand and act upon
diverse forms of well-being comprehensively. Goingforward, we will continue to research human information science to understand and improve people’s
well-being from various viewpoints including psychology, neuroscience, information science, philoso-[5]

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

[6]
[7]

A. Lutz, H. A. Slagter, J. D. Dunne, and R. J. Davidson, “Attention
Regulation and Monitoring in Meditation,” Trends in Cognitive Sciences, Vol. 12, pp. 163–169, 2008. https://doi.org/10.1016/
j.tics.2008.01.005
Y. Ooishi, M. Fujino, V. Inoue, M. Nomura, and N. Kitagawa, “Differential Effects of Focused Attention and Open Monitoring Meditation on Autonomic Cardiac Modulation and Cortisol Secretion,”
Frontiers in Physiology, Vol. 12, Article 675899, July 2021. https://doi.org/10.3389/fphys.2021.675899
J. Watanabe and A. Murata, “Exploring Positive Computing in Your
Own Way: Approach to Wellbeing with Embodiment,” Journal ofJapan Society of Kansei Engineering, Vol. 18, No. 2, pp. 63–67, 2020
(in Japanese).
J. Watanabe and D. Chen (compilers and editors), H. Ando, K.
Sakakura, and A. Murata (editors), “Towards Collective Wellbeing—
Its Concept, Practice, and Technology,” BNN Inc., 2020 (in Japanese).
“Our Well-being Cards,” ICC Kids’ Program, 2021 Edition (in Japanese).
https://hyper.ntticc.or.jp/kids2021/ourwellbeingcards
J. Watanabe, A. Murata, C. Takayama, M. Nakatani, and Y. Deguchi,“Evaluation of Self-as-We—Development of Self-as-We Measures—,” PROSPECTUS, Vol. 20, pp. 1–14, 2020 (in Japanese).
A. Murata, J. Watanabe, and Y. Deguchi, “Relationship between
Trends in Depression under the COVID-19 Pandemic and ‘Self-asWe,’” PROSPECTUS, Vol. 20, pp. 15–33, 2020 (in Japanese).

42Feature Articles

Naoki Saijo

Senior Research Scientist, Supervisor, Sensory
Resonance Research Group Leader, Human
Information Science Laboratory, NTT Communication Science Laboratories.
He received a B.E. and M.E. in mechanicalengineering from Waseda University, Tokyo, in1999 and 2001, and Ph.D. from Tokyo Instituteof Technology in 2010. He joined NTT in 2001
and has been engaged in research on human
information processing, including mechanismsof human sensorimotor control and motor learning. He has recently been focused on the interaction mechanisms of human emotion and action,
especially in highly skilled people such as athletes and performers. He is a member of theSociety for Neuroscience (SfN), the Japan Neuroscience Society (JNS), and the Institute ofElectronics, Information and Communication
Engineers (IEICE).

Masahiro Fujino

Research Specialist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
He received a B.B. from Kobe University,
Hyogo, in 2002, and B.Ed., M.Ed., and Ph.D. ineducation from Kyoto University in 2014, 2016,
and 2019. He joined NTT in 2021 and has been
engaged in research on human information processing, including mechanisms of human attention control and emotion regulation induced by
mindfulness meditation. He has recently studied
the mechanism and effect of observing natural
breathing on well-being. He is a member of theJapanese Psychological Association (JPA), Japanese Association of Mindfulness, and Japanese
Association for the Study of Buddhism and Psychology.

Yuuki Ooishi

Senior Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
He received a B.S., M.S., and Ph.D. in physics
from the University of Tokyo in 2003, 2005, and2008. He joined NTT in 2008 and has been
engaged in research on human information processing, including mechanisms of human emotion induced by sounds or music. He has recently
studied the mechanisms of human emotion,
including the effects of mindfulness meditation,
parents-infant interaction, and auditory-tactileinteraction. He is a member of JNS.

Junji Watanabe

Senior Distinguished Researcher, Sensory
Resonance Research Group, Human Information
Science Laboratory, NTT Communication Science Laboratories.
He received a Ph.D. in information science andtechnology from the University of Tokyo in 2005
and joined NTT in 2013. His research is focusedon haptic communication and collective wellbeing. He concurrently works at NTT Social
Informatics Laboratories and NTT Human Informatics Laboratories.

Aiko Murata

Research Scientist, Sensory ResonanceResearch Group, Human Information Science
Laboratory, NTT Communication Science Laboratories.
She received a B.A., M.A., and Ph.D. in humansciences from Hokkaido University in 2009,2011, and 2016. She joined NTT in 2018 and has
been engaged in research on human information
processing, including mechanisms of humanempathy and well-being. She has recently studied the interaction mechanisms of human emotions and personal and social well-being using
subjective and physiological measures. She is amember of JPA and Japan Cognitive Science
Society (JCSS).

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

43

Feature Articles: Adapting to the Changing Present
and Creating a Sustainable Future

Communication Science that Adapts
to the Changing Present and Createsa Sustainable Future—Aiming toCreate Technology that Brings
Harmony and Symbiosis among
People, Society, and the Environment
Futoshi Naya
Abstract

NTT Communication Science Laboratories (CS Labs) celebrated its 30th anniversary in 2021. Fromthe beginning, we have been engaged in research to construct basic theories that address the essence ofhuman and information sciences and develop innovative technologies that can bring about changes insociety. We aim to achieve communication that “reaches the heart” between humans and between
humans and computers. In this article, I present the recent research results at CS Labs and efforts toward
future advances.
Keywords: communication science, artificial intelligence, brain science

1. Introduction
NTT Communication Science Laboratories (CSLabs) was established on July 4, 1991, at Keihanna
Science City, Kyoto. CS Labs, which was initiallyhoused within the Advanced Telecommunications
Research Institute International (ATR) and started
from two research groups on machine learning andinformation theory, currently has about 150 researchers at Keihanna and Atsugi, Kanagawa Prefecture.
Along with last year’s open house, we set up a website to commemorate the 30th anniversary of our
founding and summarize our main research results[1].
Research themes at CS Labs, which began with themission of understanding human-to-human communication, have been developed in the two areas. The
first is approaching and surpassing human abilities
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

including media-information processing enabling
computers to have the same abilities as humans, i.e.,seeing, listening, and speaking, as well as quantum
information theory and machine learning. The second
area is deeply understanding humans including
human information science for revealing various
human perception and motor skills and brain science
focusing on cognitive flexibility and diversity for
elucidating excellent cognitive ability such as thoseof top athletes (Fig. 1).
Although these research areas have developed andchanged to reflect the technological progress andsocial needs of each era over 30 years, we have continued to develop technologies that are close to people and society while adhering to the philosophy ofunderstanding the essence of communication. I present our recent research results in the following areas.

44

Feature Articles

Human information science

Deeply understanding
humansBrain science focusing on cognitive
flexibility and diversity
Fast ballSlow ball

Dog
Understanding
Speaking
Fast ballSlow ballBall impact

Swing start

26.1
monthsCommunication thatreaches the heart

Age in month

Data science andmachine learning

Media processing

Approaching and surpassing
human abilities

Fig. 1. Research areas of CS Labs.

2. Technologies that approach and surpass
human abilities
With the advent of deep learning, artificial intelligence (AI) technology related to media processing,such as seeing, listening, and speaking, continues tomake remarkable progress, and has already exceeded
human discrimination performance in, for example,
image-recognition competitions. However, to obtainhigh performance in deep learning, a huge amount oftraining data consisting of pairs of input data and theircorrect labels is generally required.
Even if we humans can obtain only a small amountof information, we can make flexible and sophisticated inferences from experience and supplementing
missing information with other information. For
example, although a photograph captures threedimensional (3D) spatial information as a 2D image,humans can infer to some extent 3D information suchas the shape and depth of the subject just by looking
at the photograph based on experience.
Conventionally, to make computers learn suchabilities, a large amount of data including 3D information measured with a special device, such as adepth sensor or stereo camera, is needed in addition
to a 2D image taken as a photograph, but this is very
costly. To overcome this problem of obtaining data,
we proposed the deep learning technology aperture
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

rendering generative adversarial network [2] thattakes into account the relationship between apertureand blur as an optical constraint of the camera. This
technology can learn 3D information such as depth
and bokeh effects from only general photographs (2D
information) such as public images on the Internet,
and generate new images with different blur andbokeh effects with different depths of field.
Humans also have the excellent ability of selective
listening, i.e., focusing on the specific voice of a person in a multi-person talking situation, such as aparty, or a noisy or reverberant environment. At CS
Labs, we proposed SpeakerBeam, a technology that
distinguishes the voice of the target person using only
voice. In 2021, we proposed Multimodal SpeakerBeam [3], which enables selective listening frommultiple clues like what humans can do by combining
video information and audio. This technology
achieves highly accurate speaker-voice separation
when there are multiple speakers with similar voice
qualities. The movement of the lips is used as themain clue, and conversely, when the image of the lips
cannot be obtained, the voice is used. This research isdeveloping into a universal sound extraction technology that can distinguish not only voice but also
notable sounds such as sirens of fire engines and thebarking of dogs. The Feature Articles in this issue
introduce the latest speech enhancement technology
45

Feature Articles

for extracting high-quality voice as if it was recordedwith a microphone near the speaker from the voicerecorded with a microphone located at a remote location [4].
Since the 1990s, CS Labs has been researching
dialogue systems that can naturally communicate
with people. We initially focused on researching dialogue systems for specific purposes such as reservations and searches, but we are now researching chatdialogue systems that can respond naturally whilehandling a wide range of topics regardless of purpose[5]. In 2021, with the cooperation of Seika Town Hallin Kyoto Prefecture, where the Keihanna CS Lab islocated, we started an AI demonstration experiment[6] in which visitors can enjoy chat dialogue while
performing operations such as government officeguidance and providing tourist information. We havealso released the largest Transformer dialogue modelin Japanese, which combines ultra-large-scale dialogue data collected on the web and deep learningtechniques [7]. We have been building a more cohesive dialogue system by discussing image scenes seenfrom the window of a vehicle where the situation of
the place changes from moment to moment alongwith the user. We are also engaged in research forconstructing a system that can memorize the preferences of the user, maintain the consistency of the
dialogue content, and continue dialogue closer tohumans.
There has also been progress in the field of machinelearning in terms of being close to people. For example, when making decisions for people such as loan
approval and recruitment, conventional machinelearning techniques that simply prioritize prediction
accuracy can result in unfair predictions when using
a variety of subtle human characteristics, such as
gender, race, and disability. We proposed a machinelearning method that is based on causal relationships[8] for achieving fair accurate prediction for eachindividual by modeling prior knowledge about
unfairness as a causal relationship (causal graph)between features and prediction results.
This issue introduces, as a cutting-edge achievement of machine learning, CS Labs’ distributed deeplearning technology for achieving signal-free mobility using digital twins with an eye on a future autonomous driving society [9]. This optimization technology improves the average moving speed of all vehicles while preventing them from colliding throughcommunicating and coordinating with one another ina road network without traffic lights.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

3. Research to deeply understand humans
At CS Labs, we have been studying various illusions caused by the latent functions of the brain in
exploring the mechanisms of human perception andmovement. We have also published a website called
Illusion Forum [10] where one can experience various visual and auditory illusions. We obtained resultsthat clarify the various latent functions of the brain
from conducting an illusion-presentation experimentusing virtual reality, indicating, for example, that abatter feels as if the ball is rising when the pitcher’s
pitching motion becomes faster even if the ball trajectories and ball speeds are exactly the same.
In October 2021, we started joint research with
Shizuoka General Hospital on voice and language
recognition for cochlear implant wearers [11]. It is
known that even deaf children can acquire the samelevel of spoken language as children with normal
hearing by wearing cochlear implants at an earlystage, but the mechanism of speech perception andlanguage development in the brain remains unclear.
By approaching this issue from both medicine andbrain science, we are attempting to elucidate aspectsof the hearing mechanism of the elderly and otherhearing-impaired people as well as the mechanisms
behind individual differences in speech perception
and language development.
From research on language acquisition, CS Labs
published a word-familiarity database that evaluatedthe degree of familiarity of about 80,000 Japanesewords on a 7-point scale in 1999. In 2021, we built
the Reiwa version [12] for re-examining the original
and includes more than 160,000 additional words.
The database is commercially provided by NTT
Printing Corporation. We also developed a method
for estimating a person’s approximate size of vocabulary by answering whether the person knows 50
selected words using this word-familiarity database.
The website is open to the public [13].
4. Toward the creation of technologies thatbring harmony and symbiosis among people,
society, and the environment
Research activities at CS Labs have been carriedout by pursuing each area of expertise, but the amountof research that crosses two or more such areas has
been increasing. For example, media-processing
research that specializes in a single modality such as
seeing or listening has evolved into cross-modal
information processing that handles multiple media
46

Feature Articles

at the same time, such as speaker identification using
sound and video, as described above. In human science as well, research is shifting to elucidating themechanism of multisensory integration. In 2021, thepitcher simulator, which is the result of research thatbrought together all the knowledge and techniques ofbrain science, human science, and media processing
at CS Labs, contributed to winning the gold medal of
the Japan Women’s Softball National Team at the
major international sporting event held in Tokyo andwas featured as a “secret weapon” [14] in newspapers
and television. As another example, this issue presents an approach to personal cardiac modeling thatcombines mobile sensing, media processing, andmachine learning [15].
Recent advances in science and technology andtheir commoditization are accelerating rapidly. Furthermore, the COVID-19 pandemic, global climatechange, increasing natural disasters, and international
conflict have dramatically changed our daily lives
and values. There is a limit to solving increasingly
complex and diversified social issues simply by pursuing research within an area of expertise, andresearch that creates new value through synergies
with research results in peripheral areas and researchin different areas is becoming increasingly importantto open up new interdisciplinary research areas.
Basic research also requires a research approachfrom a long-term perspective that is fundamentallydifferent from the past. Therefore, the Institute for
Fundamental Mathematics was established in October 2021 to study the fundamental theory of modern
mathematics at CS Labs [16]. For more informationon our research at the Institute for Fundamental
Mathematics, please see the article in this issue [17].
Toward the implementation of IOWN (the Innovative
Optical and Wireless Network), while making fulluse of the latest mathematical methods, we will
tackle and solve problems that occur in various areasand continue to work on communication science
research to develop technologies that bring harmonyand symbiosis among people having diverse values,
society, and the environment.
References[1][2]

Website of 30th anniversary of NTT Communication Science Laboratories (in Japanese), https://www.kecl.ntt.co.jp/30th/
T. Kaneko, “Unsupervised Depth and Bokeh Learning from Natural

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Images Using Aperture Rendering Generative Adversarial Networks,”
NTT Technical Review, Vol. 20, No. 7, pp. 89–95, July 2022. https://doi.org/10.53829/ntr202207ra1[3] M. Delcroix, T. Ochiai, H. Sato, Y. Ohishi, K. Kinoshita, T. Nakatani,and S. Araki, “Developing AI that Pays Attention to Who You Want to
Listen to: Deep-learning-based Selective Hearing with SpeakerBeam,” NTT Technical Review, Vol. 19, No. 9, pp. 39–45, 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202109fa4.html[4] T. Nakatani, R. Ikeshita, N. Kamo, K. Kinoshita, S. Araki, and H.
Sawada, “AI Hears Your Voice as if It Were Right Next to You—Audio
Processing Framework for Separating Distant Sounds with Closemicrophone Quality,” NTT Technical Review, Vol. 20, No.10, pp.
49–55, Oct. 2022.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202210fa7.html[5] H. Sugiyama, M. Mizukami, T. Arimoto, H. Narimatsu, Y. Chiba, and
H. Nakajima, “The Day a System Becomes a Conversation Partner—
Exploring New Horizons in Social Dialogue Systems with Largescale Deep Learning,” NTT Technical Review, Vol. 19, No. 9, pp.
28–33, 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202109fa2.html[6] NTT press release issued on Nov. 12, 2021 (in Japanese).
https://group.ntt/jp/topics/2021/11/12/ai_seika.html[7] NTT press release issued on Sept. 30, 2021 (in Japanese).
https://group.ntt/jp/topics/2021/09/30/transformer.html[8] Y. Chikahara, S. Sakaue, A. Fujino, and H. Kashima, “Learning Individually Fair Classifier with Path-specific Causal-effect Constraint,”
Proc. of the 24th International Conference on Artificial Intelligenceand Statistics (AISTATS), Virtual, Aug. 2021.
[9] K. Niwa, “Smart Traffic Coordination via Learnable Digital Twins—
Future Possibilities of Distributed Deep Learning,” NTT Technical
Review, Vol. 20, No. 10, pp. 61–66, Oct. 2022.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202210fa9.html[10] Website of Illusion Forum by CS Labs (in Japanese), https://illusionforum.ilab.ntt.co.jp/[11] NTT press release issued on Oct. 19, 2021 (in Japanese).
https://group.ntt/jp/newsrelease/2021/10/19/211019a.html[12] NTT press release issued on June 3, 2020 (in Japanese).
https://group.ntt/jp/newsrelease/2020/06/03/200603a.html[13] Website of “Reiwa-version of Vocabulary Test” (in Japanese), https://www.kecl.ntt.co.jp/icl/lirg/resources/goitokusei/vocabulary_test/php/
login.php[14] M. Yamaguchi, D. Nasu, D. Mikami, T. Kimura, T. Fukuda, and M.
Kashino, “Women’s Softball × Sports Brain Science,” NTT Technical
Review, Vol. 20, No. 2, pp. 32–38, Feb. 2022.
https://doi.org/10.53829/ntr202202fa3[15] K. Kashino, R. Shibue, and S. Tsukada, “Personal Heart Modeling
Using Mobile Sensing,” NTT Technical Review, Vol. 20, No. 10, pp.
56–60, Oct. 2022.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202210fa8.html[16] NTT press release, “NTT has established the Institute for Fundamental Mathematics—Advancing the pace of exploration into the unexplored principles of quantum computing,” Oct. 1, 2021.
https://group.ntt/en/newsrelease/2021/10/01/211001a.html[17] M. Wakayama, “Number Theory and Quantum Physics Based on
Symmetry—Themes from Quantum Optics,” NTT Technical Review,Vol. 20, No. 10, pp. 67–76, Oct. 2022.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202210fa10.html

47

Feature Articles

Futoshi Naya

Vice President and Head of NTT Communication Science Laboratories.
He received a B.E. in electrical engineering,
M.S. in computer science, and Ph.D. in engineering from Keio University, Kanagawa, in 1992,
1994, and 2010. He joined NTT Communication
Science Laboratories in 1994. From 2003 to
2009, he was with Intelligent Robotics and Communication Laboratories, Advanced Telecommunications Research Institute International(ATR). His research interests include communication robots, sensor networks, pattern recognition, data mining in cyber-physical systems, andAI-based tailor-made education support. He is amember of the Institute of Electrical and Electronics Engineers (IEEE), the Robotics Societyof Japan, the Society of Instrument and Control
Engineers, and the Institute of Electronics, Information and Communication Engineers (IEICE).

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

48

Feature Articles: Adapting to the Changing Present
and Creating a Sustainable Future

AI Hears Your Voice as if It Were
Right Next to You—Audio Processing
Framework for Separating DistantSounds with Close-microphone
Quality
Tomohiro Nakatani, Rintaro Ikeshita, Naoyuki Kamo,
Keisuke Kinoshita, Shoko Araki, and Hiroshi Sawada
Abstract

When we capture speech using microphones far from a speaker (distant microphones), reverberation,
voices from other speakers, and background noise become mixed. Thus, the speech becomes less intelligible, and the performance of automatic speech recognition deteriorates. In this article, we introduce
the latest speech-enhancement technology for extracting high-quality speech as if it were recorded using
a microphone right next to the speaker (close microphone) from the sound captured using multiple distant microphones. We discuss a unified model that enables dereverberation, source separation, anddenoising in an overall optimal form, switch mechanism that enables high-quality processing with asmall number of microphones, and their integration with deep learning-based speech enhancement (e.g.,
SpeakerBeam).
Keywords: speech enhancement, microphone array, far-field speech recording

1. Introduction
High-quality speech applications using a microphone placed near the speaker’s mouth (close microphone) have been widely used, such as automatic
speech recognition (ASR) using a smartphone andremote conferencing using a headset. For artificial
intelligence (AI) to become a more practical assistantin our daily lives, it is required to handle speech in thesame manner even when the speech is captured using
microphones far from the speaker (distant microphones). However, with distant microphones, reverberation reflected from the walls or ceilings, voicesfrom other speakers, and background noise become
mixed. Therefore, the quality of the captured speech
deteriorates significantly, and the performance ofNTT Technical Review

Vol. 20 No. 10 Oct. 2022

speech applications, such as ASR, greatly degrades.
To solve this problem, we are developing speechenhancement technology for extracting a high-quality
voice of each speaker as if it were captured with aclose microphone from sound captured with distantmicrophones. This article introduces the latest technology for multi-microphone speech enhancementthat uses multiple microphones for achieving higher
quality processing than with a single microphone.
2. Challenges for achieving
close-microphone quality
To extract a speech with close-microphone quality
from sound captured using distant microphones, it isnecessary to achieve three types of processing:49

Feature Articles

Capturedsound

Dereverberation

Sourceseparation

Inversion
Reverberationprocessestimation

DenoisingInversion
Sound-mixing
process
estimation

EnhancedspeechInversion
Noise-mixing
process
estimation(a) Conventional multi-microphone speech enhancement

Capturedsound

Dereverberation

Sourceseparation

Inversion

DenoisingInversion

EnhancedspeechInversion

Captured-sound-generation-process estimation
based on unified criterion

Close-mic.
signalmodel(b) Unified model-based multi-microphone speech enhancement

Fig. 1. Conventional and unified model-based methods for multi-microphone speech enhancement.

dereverberation, source separation, and denoising.
Dereverberation transforms a blurry speech with adistant impression into a clear speech with theimpression of being right next to the microphone.
When multiple speakers’ voices and backgroundnoise are mixed, they are separated into individual
sounds by source separation and denoising. This
makes it possible to extract each speaker’s voice withclose-microphone quality.
Conventional multi-microphone speech-enhancement methods achieve dereverberation, source separation, and denoising by estimating the generation
processes of captured sound, in which sounds propagate from the sources to the microphones and mix,
then applying the inverse of the estimated processes
to the captured sound (Fig. 1(a)). Specifically, theprocesses of reverberation reflecting from walls or
ceilings and reaching the microphones, multiple
sounds coming from different directions and mixing,
and noise coming from all directions and mixing are
estimated, then their inversions are applied.
For example, WPE (weighted prediction error) [1]developed by NTT is the world’s first dereverberation
method. It can achieve almost perfect dereverberation
by estimating the reverberation process of the captured sound without any prior knowledge on whatenvironments in which the sound was captured (i.e.,by blind processing), provided the captured sound
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

does not contain noise. Independent component
analysis [2, 3], which has been actively studied
worldwide by researchers, including NTT, canachieve precise source separation by blind processing, provided the captured sound does not containreverberation.
However, these conventional multi-microphone
speech-enhancement methods cannot be used tosolve the three problems (reverberation, multiple
sound sources, and noise) at the same time in an overall optimal form. It is impossible to simultaneously
estimate all generation processes from the captured
sound, which is a mixture of noise, reverberation, andmultiple sounds, and execute the inversion of theentire process. Therefore, we have to apply each process in turn. For example, dereverberation is executed
first assuming that noise is absent, so precise dereverberation is impossible. We then apply sound-source
separation and denoising, assuming that reverberation is wholly suppressed; thus, the best performancecannot be achieved. It is therefore impossible toachieve overall optimal speech enhancement whencombining these conventional methods.
The sound captured using distant microphones
almost always contains reverberation, multiple soundsources, and noise. For this reason, it has been considered critical to optimally apply the three types ofprocessing, dereverberation, source separation, and50

Feature Articles

Word accuracy
60%(a) Conventional 1:
source separation+ denoising

70%

80%

Word accuracyof captured signal
37.5%

65.6%

(b) Conventional 2:
dereverberation+ source separation+ denoising

90%

76.5%

83.7%

(c) Unified model

86.8%(d) Deep learning+ unified modelFig. 2. Improvement in ASR performance using multi-microphone speech enhancement.

denoising, in an overall optimal form.
3. Unified model for dereverberation, source
separation, and denoising
In response to this, we devised a unified model thatcan solve the three problems in an overall optimal
form [4, 5]. The unified model first mathematicallymodels the general properties that close-microphonequality speech and noise must satisfy. It can thenenable overall optimum processing by optimizing
each type of processing on the basis of the unified
criterion that the sound obtained from combining thethree types of processing best satisfies the closemicrophone property (Fig. 1(b)). For example, we
can significantly improve ASR using distant microphones with the unified model (Figs. 2(a)–(c)).
Figure 3 shows a spectrogram of two speech signals and noise captured using a close microphone andthe mixture of them captured using a distant microphone. The speech signals captured using the close
microphone are sparse signals in which the soundconcentrates in separate local areas, and are non-stationary signals that change with time. In contrast,
noise is a dense and stationary signal in which thesound spreads over a wider area and does not change
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

much with time. However, the mixture captured using
a distant microphone has different characteristics. It
is denser than the speech signals with close-microphone quality and more non-stationary than noisewith close-microphone quality.
The unified model uses the differences in thesesound characteristics. It controls dereverberation,
source separation, and denoising so that the soundresulting from their application best satisfies the characteristics of speech and noise with close-microphonequality. For example, in dereverberation, we estimate
the reverberation-generation process and apply its
inversion so that the sound obtained in combination
with source separation and denoising best satisfies
the close-microphone quality. Similarly, we optimize
source separation and denoising by estimating thesound-generation process and applying its inversion
to best satisfy the close-microphone quality whencombined with dereverberation. With the aim ofachieving close-microphone quality, it has become
possible to execute overall optimum processing whencombining all types of processing.
We have also developed computationally efficientalgorithms for unified model-based multiple microphone speech enhancement [6, 7]. For example, theprocessing using the unified model illustrated in51

Feature Articles

Sound with close-microphone quality
Speech 2

NoiseFrequency

Speech 1

Sound captured using a
distant microphone
Dense andnon-stationary

Sparse and non-stationary

Dense and stationary

Time

Unified model:
Controls each process
so that enhanced sounds
best satisfy above
characteristics

Frequency

Sparse and non-stationary

Time
Spectrograms: power of sounds at each time frequency point (red: high, blue: low)

Fig. 3. Spectrograms of sounds captured using close and distant microphones.

Fig. 2 (executing overall optimization of dereverberation, source separation, and denoising using eightmicrophones) can now be completed in real timeusing a Linux computer. When we limit the problem
to extracting a speaker’s voice by blind processing
from background noise and little reverberation, we
can reduce the computational cost to the extent that
real-time processing is possible even with an embedded device.
4. Switch mechanism enabling accurate
estimation with a smaller numberof microphones
A switch mechanism is an applied technology
using the unified model and enables highly accurate
estimation even with a relatively small number ofmicrophones [8, 9]. With conventional multi-microphone speech-enhancement methods, it is necessary
to use a sufficiently large number of microphones for
precise processing compared with the number ofsound sources included in the captured sound. Thishinders the application of multi-microphone speechenhancement to real-life problems. To solve thisproblem, we introduce a switch mechanism that can
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

improve estimation accuracy with a small number ofmicrophones.
The idea of this switch mechanism is summarized
as follows. Even when the captured sound contains
many sound sources, the number of sources appearing simultaneously can be smaller when counting
them within each short time interval. Let us explain
this using Fig. 4. The horizontal axis is time, and ahorizontal bar in each color represents when each of
the three speakers speaks. When we divide the horizontal axis into short intervals a, b, and c, as shown in
the figure, only two speakers are speaking in eachtime interval even though there are three speakers intotal. With this interval division, we can improve
multi-microphone speech enhancement by applying
it separately to each short interval with the decreased
number of speakers. We call this a switching mechanism because we switch speech enhancement foreach short interval.
When combined with the unified model, the switch
mechanism can perform best. We can use the unified
model to optimize the interval-wise application of
speech enhancement and the switch mechanism’s
time interval division. This unified model-based
speech enhancement can optimize the all processing
52

Feature Articles

Interval aInterval b

Interval cInterval b

Interval a

Speaker A
Speaker B
Speaker C

Time
Less than two speakers within each interval; a, b, and c.

Fig. 4. An example of each speaker’s utterance periods in a conversation among three speakers.

types (dereverberation, source separation, anddenoising) with the switch mechanism so that theenhanced speech best satisfies the close-microphonequality.
5. Unified model as a versatile technique ofaudio-signal processing
As described above, our unified model provides
theoretically and practically excellent guidelines for
integrating the three processing types in speechenhancement that we have conventionally combinedin more heuristic ways. The unified model can provide a mechanism to achieve overall optimizationeven when combining more complicated processing
approaches such as the switch mechanism. We canuse the unified model as a versatile technique providing a basis for future audio-signal processing-technology development.
6. Future direction: optimal integration withdeep learning
Deep learning is another fundamental approach tospeech enhancement, and its integration with multimicrophone speech-enhancement methods is vital forthe future development. While deep learning canconduct processing that is difficult with multi-microphone speech enhancement, such as voice characteristics-based selective listening using SpeakerBeam, adeep learning-based approach for computational
selective hearing based on the characteristics of thetarget speaker’s voice [10], it also has severe limitations. For example, with deep learning-based speechenhancement, improvement in ASR performance isminimal, and sound quality largely degrades due toreverberation. Therefore, both deep learning andmulti-microphone speech-enhancement methods
complement each other, thus are indispensable. For
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

example, even when the ASR performance or qualityof enhanced speech does not much improve solely bydeep learning-based speech enhancement, they canbe improved when combined with the multi-microphone speech enhancement. Figure 2(d) shows thatthe combined approach further improves ASR performance compared with solely using the unified modelbased multi-microphone speech enhancement.
Speech enhancement will have much higher functionality and quality through developing an optimal
integration method for both deep learning and multimicrophone speech enhancement.
References
[1]

[2]

[3]

[4]

[5]
[6]
[7]

[8]
[9]

T. Nakatani, T. Yoshioka, K. Kinoshita, M. Miyoshi, and B.-H. Juang,“Speech Dereverberation Based on Variance-normalized Delayed
Linear Prediction,” IEEE Trans. Audio, Speech, and Language Process., Vol. 18, No. 7, pp. 1717–1731, 2010.
N. Ono and S. Miyabe, “Auxiliary-function-based Independent Component Analysis for Super-Gaussian Sources,” LVA/ICA 2010: Latent
Variable Analysis and Signal Separation, pp. 165–172, Springer,
2010.
H. Sawada, S. Araki, and S. Makino, “Underdetermined Convolutive
Blind Source Separation via Frequency Bin-wise Clustering and Permutation Alignment,” IEEE Trans. Audio, Speech, and Language
Process., Vol. 19, No. 3, pp. 516–527, 2011.
T. Nakatani, C. Boeddeker, K. Kinoshita, R. Ikeshita, M. Delcroix,
and R. Haeb-Umbach, “Jointly Optimal Denoising, Dereverberation,and Source Separation,” IEEE/ACM Trans. Audio, Speech, and Language Process., Vol. 28, pp. 2267–2282, 2020.
R. Ikeshita and T. Nakatani, “Independent Vector Extraction for Fast
Joint Blind Source Separation and Dereverberation,” IEEE Signal
Process. Lett., Vol. 28, pp. 972–976, 2021.
R. Ikeshita, T. Nakatani, and S. Araki, “Block Coordinate Descent
Algorithms for Auxiliary-function-based Independent Vector Extraction,” IEEE Trans. Signal Process., Vol. 69, pp. 3252–3267, 2021.
T. Ueda, T. Nakatani, R. Ikeshita, K. Kinoshita, S. Araki, and S.
Makino, “Low Latency Online Source Separation and Noise Reduction Based on Joint Optimization with Dereverberation,” Proc. of the
29th European Signal Processing Conference (EUSIPCO 2021), pp.
1000–1004, Dublin, Ireland, 2021.
R Ikeshita, N. Kamo, and T. Nakatani, “Blind Signal Dereverberation
Based on Mixture of Weighted Prediction Error Models,” IEEE Signal
Process. Lett., Vol. 28, pp. 399–403, 2021.
T. Nakatani, R. Ikeshita, K. Kinoshita, H. Sawada, N. Kamo, and S.
Araki, “Switching Independent Vector Analysis and Its Extension to53

Feature Articles

Blind and Spatially Guided Convolutional Beamforming Algorithms,” IEEE/ACM Trans. Audio, Speech, and Language Process.,
Vol. 30, pp. 1032–1047, 2022.
[10] M. Delcroix, K. Zmolikova, K. Kinoshita, S. Araki, A. Ogawa, and T.
Nakatani, “SpeakerBeam: A New Deep Learning Technology for

Tomohiro Nakatani

Senior Distinguished Researcher, Signal Processing Research Group, Media Information
Laboratory, NTT Communication Science Laboratories.
He received a B.E., M.E., and Ph.D. from
Kyoto University in 1989, 1991, and 2002. Sincejoining NTT as a researcher in 1991, he has been
investigating speech-enhancement technologies
for developing intelligent human-machine interfaces. He was a visiting scholar at Georgia Institute of Technology in 2005 and visiting assistantprofessor in the Department of Media Science,
Nagoya University, Aichi from 2008 to 2018. Hereceived the 2005 Institute of Electronics, Information and Communication Engineers (IEICE)Best Paper Award, the 2009 Acoustical Societyof Japan (ASJ) Technical Development Award,
the 2012 Japan Audio Society Award, the 2015
Institute of Electrical and Electronics Engineers(IEEE) Automatic Speech Recognition andUnderstanding Workshop (ASRU) Best Paper
Award Honorable Mention, the 2017 Maejima
Hisoka Award, and the 2018 International Workshop on Acoustic Echo and Noise Control(IWAENC) Best Paper Award. He was a memberof the IEEE Signal Processing Society Audio andAcoustic Signal Processing Technical Committee (AASP-TC) from 2009 to 2014 and served as
the chair of the AASP-TC Review Subcommittee
from 2013 to 2014. He was a member of theIEEE Signal Processing Society Speech andLanguage Processing Technical Committee (SLTC) from 2016 to 2021. He served as an associate
editor of the IEEE Transactions on Audio,
Speech and Language Processing from 2008 to
2010, chair of the IEEE Kansai Section Technical
Program Committee from 2011 to 2012, technical program co-chair of IEEE Workshop on
Applications of Signal Processing to Audio andAcoustics (WASPAA) 2007, workshop co-chairof the 2014 REVERB Challenge Workshop,
general co-chair of the 2017 IEEE ASRU Workshop, and chair of IEEE Signal Processing Society Kansai Chapter from 2019 to 2020. He is anIEEE Fellow, and a member of IEICE and ASJ.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Extracting Speech of a Target Speaker Based on the Speaker’s Voice
Characteristics,” NTT Technical Review, Vol. 16, No. 11, pp. 19–24,
2018.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201811fa2.html

Rintaro Ikeshita

Researcher, Signal Processing ResearchGroup, Media Information Laboratory, NTT
Communication Science Laboratories.
He received a B.E. and M.S. from the University of Tokyo in 2013 and 2015 and joined NTT
in 2018. He received the 17th Itakura Prize Innovative Young Researcher Award from ASJ in
2022 and the 49th Awaya Prize Young Researcher Award from ASJ in 2021. He is a member ofIEEE and ASJ.

Naoyuki Kamo

Researcher, Signal Processing ResearchGroup, Media Information Laboratory, NTT
Communication Science Laboratories.
He received a B.S. and M.S. from Kyoto University in 2012 and 2014. Before he joined NTT
in 2020, he developed as a contributor the wellknown deep learning-based open software tools
for speech-signal processing ESPNET-1 and 2.
Since joining NTT, he has developed neural network-based techniques, such as speech enhancement and intelligibility prediction. He is a member of ASJ.

54

Feature Articles

Keisuke Kinoshita

Research Scientist, Google Japan.
He received an M. Eng. and Ph.D. from Sophia
University, Tokyo, in 2003 and 2010. After joining NTT Communication Science Labs in 2003,
he was engaged in fundamental research on various types of speech, audio, and music-signal
processing, including 1ch/multi-channel speechenhancement (blind dereverberation, source separation, noise reduction), speaker diarization,robust speech recognition, and distributed
microphone-array processing, and developed
several types of innovative commercial software.
He has been in his current position since October
2022. He has been serving as an associate editorof IEEE/ACM Transactions on Audio, Speechand Language Processing since 2021 and member of IEEE AASP-TC since 2019. He also
served as the chief coordinator of the REVERB
Challenge (2014), an editor of IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences (from 2013 to2017), and guest editor of EURASIP journal on
advances in signal processing (2015). He washonored to receive the 2006 IEICE Paper Award,
the 2010 ASJ Outstanding Technical Development Prize, the 2011 ASJ Awaya Prize, the 2012
Japan Audio Society Award, 2015 IEEE-ASRU
Best Paper Award Honorable Mention, and 2017
Maejima Hisoka Award. He is a member of IEEEand ASJ.

Hiroshi Sawada

Executive Manager and Senior Distinguished
Researcher, Innovative Communication Laboratory, NTT Communication Science Laboratories.
He received a B.E., M.E., and Ph.D. in information science from Kyoto University in 1991,
1993, and 2001. His research interests include
statistical signal processing, audio-source separation, array-signal processing, machine learning, latent variable modeling, graph-based data
structures, and computer architecture. He
received the Best Paper Award of the IEEE Circuit and System Society in 2000 and the Best
Paper Award of IEEE Signal Processing Society
in 2014. He is an IEEE Fellow and member of
IEICE and ASJ.

Shoko Araki

Group Leader and Senior Research Scientist,
Signal Processing Research Group, Media Information Laboratory, NTT Communication Science Laboratories.
She received a B.E. and M.E. from the University of Tokyo in 1998 and 2000 and Ph.D. from
Hokkaido University in 2007. Since she joined
NTT in 2000, she has been engaged in researchon acoustic-signal processing, array-signal processing, blind-source separation, meeting diarization, and auditory scene analysis. She was amember of the IEEE Signal Processing Society
AASP-TC (2014–2019) and currently serves as
its vice chair. She has been a board member of
ASJ since 2017 and currently serves as vicepresident of ASJ (2021–2022). She also served asa member of the organizing committee of International Symposium on Independent Component
Analysis and Blind Signal Separation (ICA)
2003, IWAENC 2003, IEEE WASPAA 2007,
Hands-free Speech Communications and Microphone Arrays (HSCMA) 2017, IEEE WASPAA
2017, IWAENC 2018, IEEE WASPAA 2021 andthe evaluation co-chair of the Signal Separation
Evaluation Campaign (SiSEC) 2008, 2010, and2011. She received the 19th Awaya Prize from
ASJ in 2001, the Best Paper Award of the
IWAENC in 2003, the TELECOM System Technology Award from the Telecommunications
Advancement Foundation in 2004 and 2014, theAcademic Encouraging Prize from IEICE in
2006, the Itakura Prize Innovative Young
Researcher Award from ASJ in 2008, the Commendation for Science and Technology by the
Minister of Education, Culture, Sports, Scienceand Technology, The Young Scientists’ Prize in2014, IEEE Signal Processing Society Best
Paper Award in 2014, and IEEE ASRU 2015 Best
Paper Award Honorable Mention in 2015. She is
an IEEE Fellow for contributions to blind-source
separation of noisy and reverberant speech signals. She is a member of IEEE, IEICE, and ASJ.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

55

Feature Articles: Adapting to the Changing Present
and Creating a Sustainable Future

Personal Heart Modeling Using
Mobile Sensing
Kunio Kashino, Ryohei Shibue, and Shingo Tsukada
Abstract

Public expectations for healthcare in daily life using digital technologies are increasing. With this inmind, we first focused our research on biomedical informatics related to the human heart. We have been
experimenting with ways to observe the acoustical and electrical signals generated by the living bodyand how to replicate, in a computer, the state and function of a person’s heart at a particular point in time.
In this article, we introduce new technologies for measuring and estimating the activity of the humanheart.
Keywords: biomedical informatics, healthcare, mobile sensing

1. Personalized biomedical modeling
Even in the fields of medicine and health, the traditional limitations of physical location, time, and distance are giving way to continuous advances in sensing, communications, artificial intelligence (AI), andother information-processing technologies. An
increasing number of people use wearable devices,such as smartwatches, for their personal health management. Further utilization of biometric informationin daily life will enable continuous healthcare, useful
not only when ill but also before and after treatment,
and personalized healthcare, not just patterned treatments for diseases. Research and development andpractical applications in this field is very active
worldwide.
As part of NTT’s Medical and Health Vision, asimulator called the Bio-Digital Twin is being developed to support risk control and improved wellness[1] by predicting future physical and mental conditions of a person. The Bio-Digital Twin is a computational model that captures a wide variety of biomedical information, from molecular and cellular scales toorgans and their interconnections, even the environment in which a person lives, expressing those connections as a network. One of the components of the
Bio-Digital Twin is personal heart modeling. Personal heart modeling means measuring the operation
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

and condition of an individual’s heart then mathematically and logically representing the person’s heart in
a computer in such a way that allows simulation
under various assumptions.
Our research team aims to obtain more detailed
biometric information than with conventional wearable devices such as smartwatches, while still
enabling measurement in daily life without any undue
burden [2].
2. New sensing
One of our prototype biometric sensing instruments
is a wearable electrocardiogram (ECG) device. Electrocardiography, as commonly used in medical institutions, uses 10 electrodes placed on the arms, legs,
and chest to obtain 12 potential differences. Smartwatches that obtain a simple ECG measuring thepotential difference just between a person’s two
hands are becoming popular. In contrast, our prototype is centered on the apical region of the chest
where the heart is closest to the rib cage as a reference
point with opposite electrode poles in three nearly
orthogonal directions to capture the electrical activityof the heart in three dimensions (Fig. 1(a)). In our
prototype, the electrodes and wiring are integrated
with a stretchable shoulder belt, making it easy to fit
to any sized adult by adjusting the shoulder and waist
56

Feature Articles

(a)

(b)

(c)

Fig. 1. W earable ECG device (a) and AI Telestethoscopes (wearable type (b), handheld type (c)). Prototypes for research
purpose.

straps [3].
Another prototype biometric sensing instrument iswhat we call the “AI Telestethoscope,” which iscapable of measuring multi-channel acoustic signals
simultaneously with an ECG. We prototyped both awearable-type (Fig. 1(b)) and handheld-type(Fig. 1(c)) AI Telestethoscope. They both enable
sensed data to be transmitted in real time to a remotelocation where a doctor or nurse can listen to thesound coming from specific locations on the bodysurface by interactively selecting them on a terminal
screen. Unlike conventional wireless electronicstethoscopes that use a single microphone, our AI
Telestethoscopes use multiple microphones to simultaneously capture sound signals at multiple locations
to capture the sounds of the heart’s activity in three
dimensions.
3. Information processing of signals to explore
inside a living body
Information processing is key to estimating theinternal state and functioning of a living body frombiomedical signals measured on the surface of the
body. The novel sensing instruments introduced in
the previous section are designed to capture precise
biometric information for personal heart modeling.
The heart works in accordance with the cyclicalactivity of heart muscle cells called cardiomyocytes.
There are two types of cardiomyocyte activity: electrical and mechanical. An ECG generally shows the
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

combined effects of the actions of many cardiomyocyte cells in the heart that can be observed as electrical potential differences on the body surface. On theother hand, heart sounds are mainly caused by themechanical activity of the heart, especially the opening and closing of the four valves inside the heart.
Abnormalities in the heart can also generate distinctive sounds, such as the vibrations caused by turbulent blood flow which can be heard as a so-called“heart murmur.”Both the bio-electrical and bio-acoustic signals thatcan be observed on the surface are the result of a mixture of many factors. It is not easy to estimate thecause backwards from the effect. That is, to understand the condition of the heart inside the body by
tracing its physical mechanism in reverse from surface observations.
We are approaching this seemingly ill-posed problem by using three-dimensional (3D) electrical andacoustical information to derive 3D location clues,
and by introducing various information-processing
techniques such as the use of physical constraints andmachine learning [4].
When testing for heart abnormalities, it is notuncommon for a typical ECG to show nothing morethan irregularity in a few waveforms. To solve thisproblem, we attempt to visualize, from the ECG
waveforms, the electrical action potential of localizedgroups of cardiomyocytes by estimating statisticalparameters related to changes in cardiomyocyte
populations (tensor ECG) [5]. Visualization of action
57

Feature Articles

S1 S2Heart soundHeart soundComponent

Component

Component

Component

Component

Component

PCUSD

Baseline method [7]Fig. 2. Estimation of S1 and S2 segments.

potential information at various locations in the heart
is expected to help with early detection of arrhythmias associated with heart failure, ischemic heartdisease, and sudden cardiac death because abnormalities can be more clearly expressed than withstandard ECGs.
We also expect to be able to estimate what sounds
are coming from which locations in the heart by capturing heart sounds in three dimensions. The locationof the sound, its tone, and its acoustic characteristics
are known to be important clues for determining thepresence and severity of heart disease. Historically,sounds have been captured on the surface of the bodyusing a manual stethoscope or phonocardiograph,and there has been no better way to listen to particularsounds within the body in a non-invasive manner.
To this end, we developed a novel oscillator decomposition technique for heart sounds that can estimate
internal sound sources from sounds observed on the
body surface, called PCUSD (Physically Constrained
Unsupervised Signal Decomposition) [6]. PCUSD
focuses on the periodicity of heart movement and themechanism of heart-sound generation. Normally, theheart moves periodically, going through four stages(cyclical phases) in sequence: S1, systolic, S2, anddiastolic, where S1 corresponds to the first sound of aheartbeat and S2 corresponds to the second sound.
Different heart valves open and close during eachstage, and their vibrations generate heart sounds. We
can assume there are multiple vibrational components to the sounds made by each valve and that their
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Table 1. S
 1 and S2 segment estimation accuracy of
PCUSD and baseline method [7]. F1 score is an
accuracy measure, the larger the better.
F1 score

PCUSD

Baseline

S1
S2

96.1
96.4

86.5
85.7

amplitudes change in accordance with the conditionof the heart during the sequential phases of the cardiac cycle. This led us to construct a probabilistic
generation model of the mechanisms of heart sounds
based on physical modeling of the heart valves.
Figure 2 shows an example of applying PCUSD toa single channel of heart sound. We examined theestimation accuracy of the S1 and S2 intervals. The
figure shows a case of mitral regurgitation with associated heart murmur. The presence of a heart murmur(in this case, between S1 and S2) generally makes it
even more difficult to estimate the heart’s condition.
The right panel shows the results from a conventional
method [7] that does not use a generation model. It
shows that it is sometimes prone to estimation errors.
In contrast, the left panel shows our PCUSD’s results,
indicating improved estimation accuracy for the sameS1 and S2 intervals. Table 1 shows this improvement
numerically.
It is straightforward to apply PCUSD to multichannel heart sounds. Figure 3 shows four acoustic
channels (Fig. 3(a)) observed on the body surface of58

Feature Articles

(a)

(b)

Fig. 3. Sounds captured on the body surface (a) and estimated in-body vibration waveforms (b).

a patient with aortic stenosis and eight internal soundsource waveforms (Fig. 3(b)) estimated with PCUSD
from the four acoustic channels. Because PCUSD
uses a physical model of valve vibration, all (oralmost all) these components are assumed to originate from the four valves of the heart. Each of thesecomponents can be heard as sound. AI analysis ofeach waveform is expected to make it possible to reliably diagnose the condition of a malfunctioning heart
valve and its progressive deterioration from heartsounds—acoustic biomedical information—captured
on the body surface.
4. Toward utilization
The new sensing and estimation technologies
described above can potentially provide a moredetailed understanding of heart activity than standard
ECG and electronic stethoscope technologies currently available. Some heart abnormalities are more
likely to cause electrocardiographic abnormalities
and be detected using an ECG. Others are more likely
to cause heart sound abnormalities and be detected
through auscultation. There is yet another set ofabnormalities detectable by comparison of the heart’s
output of electrical and acoustic signals.
We aim to actualize our vision of personal heartmodeling by integrating our new technologies to estimate the condition of the heart based on real-time
biomedical data capture and information processing
NTT Technical Review

Vol. 20 No. 10 Oct. 2022with other relevant patient information such as theirelectronic health records, family medical history,
recent blood tests, pre-existing medical imaging, andcurrent public health status. This approach can potentially help people adopt a healthier lifestyle more
appropriate for their personal risks of heart disease
and lead to earlier detection of any heart abnormalities if they occur.
When discussing the usefulness of new technologies for medical and healthcare applications, solid
research and validation is of paramount importance.
Therefore, we are collaborating with multiple medical institutions, research institutions, and specialized
hospitals to assess the potential of our technologies tomeaningfully improve early detection of certain typesof heart diseases, provide useful support for people
suffering from heart failure, serve as a tool forpatients and doctors during cardiac rehabilitation.
Together with our collaborators, we will continue toactively verify the feasibility of this ongoing research.
References
[1]

[2]

Press release issued by NTT, “Medical and Health Vision: Realizationof a Bio Digital Twin—NTT Corporation Working on the ‘Future of
Medicine’ to Predict Diseases, and Mental and Physical Conditions,
Enabling People to Have Health and Hope for the Future,” Nov. 17,2020.
https://group.ntt/en/newsrelease/2020/11/17/201117c.html
M. Nakano, R. Shibue, K. Kashino, S. Tsukada, and T. Ogasawara,“New Technology for Measurement and Analysis of BiologicalSounds and Electrocardiographic Signals—Toward Early Detection

59

Feature Articles

[3]

[4]

of Heart Disease and Rehabilitation by Using Personal Heart Modeling,” NTT Technical Review, Vol. 19, No. 7, pp. 40–45, 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202107fa5.html
Y. Tsukada, M. Tokita, H. Murata, Y. Hirasawa, K. Yodogawa, Y.
Iwasaki, K. Asai, W. Shimizu, N. Kasai, H. Nakashima, and S. Tsukada, “Validation of Wearable Textile Electrodes for ECG Monitoring,” Heart Vessels, Vol. 34, No. 7, pp. 1203–1211, 2019.
M. Nakano, R. Shibue, K. Kashino, S. Tsukada, and H. Tomoike,“Gaussian Process with Physical Laws for 3D Cardiac Modeling,”
Proc. of the 28th European Signal Processing Conference (EUSIPCO
2020), pp. 1452–1456, Amsterdam, Netherlands, Jan. 2020.

Kunio Kashino

Senior Distinguished Researcher, NTT Communication Science Laboratories and NTT Basic
Research Laboratories.
He received a Ph.D. in electrical engineeringfrom the University of Tokyo in 1995, and thenjoined NTT Basic Research Laboratories. He
was a visiting scholar in Cambridge University in
2002. His research interests include audio andcrossmodal information processing, media
search, and biomedical informatics. He is a member of the Institute of Electrical and Electronics
Engineers (IEEE) and the Association for Computing Machinery. He received IEEE Transactions on Multimedia Paper Award in 2004, Maejima Hisoka Award in 2010, Kenjiro Takayanagi
Achievement Award in 2016, the Institute ofElectronics, Information and Communication
Engineers (IEICE) Achievement Award in 2017,
and the Commendation for Science and Technology by the Minister of Education, Culture,
Sports, Science and Technology in 2019.

Ryohei Shibue

[5]
[6]

[7]

S. Tsukada, “Wearable Textile Electrodes for Long-term Vector ECG
Monitoring ‘Tensor Cardiography,’” Proc. of the 14th International
Symposium on Medical ICT (ISMICT 2020), Nara, Japan, May 2020.
R. Shibue, M. Nakano, T. Iwata, K. Kashino, and H. Tomoike, “Unsupervised Heart Sound Decomposition and State Estimation with
Generative Oscillation Models,” Proc. of the 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology
Society (EMBC 2021), pp. 5481–5487, Virtual, Nov. 2021.
C. D. Papadaniil and L. J. Hadjileontiadis, “Efficient Heart Sound
Segmentation and Extraction Using Ensemble Empirical Mode
Decomposition and Kurtosis Features,” IEEE J. Biomed. Health
Inform., Vol. 18, No. 4, pp. 1138–1152, 2014.

Shingo Tsukada

NTT Fellow, Bio-Medical Informatics ResearchCenter, NTT Basic Research Laboratories.
He received an M.D. from Toyama Medical
and Pharmaceutical University and his medical
license in 1990. He also received a Ph.D. inmedicine from University of Tsukuba in 2003.
He was a visiting researcher at the University ofCalifornia San Diego, USA, from 2003 to 2005.
He joined NTT Basic Research Laboratories in2010 as a research specialist and became a seniorresearch scientist in 2013. Since then, he has
been engaged in the study of mechanism andactivity control of signal transduction of brain
cells. His current interests are the detection of
biomedical signals using novel wearable-type
and implant-type bioelectrodes based on thecomposites of conductive polymers with various
fibers and textiles. He was designated as a fellowof NTT in 2018. He is a member of Society for
Neuroscience, the Physiological Society ofJapan, the Japan Society of Applied Physics, theJapan Neuroscience Society, the Japanese Circulation Society, and the Japanese Orthopedic
Association.

Researcher, NTT Communication Science
Laboratories and NTT Basic Research Laboratories.
He received a B.E. and M.E. in information
science and technology from the University ofTokyo in 2015 and 2017. He joined NTT Communication Science Laboratories in 2017 andstudied statistical modeling for eye movements.
He is currently studying cardiovascular signal
processing. He is a member of the Japan Statistical Society and the Japan Neuroscience Society.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

60

Feature Articles: Adapting to the Changing Present
and Creating a Sustainable Future

Smart Traffic Coordination via
Learnable Digital Twins—Future
Possibilities of Distributed Deep
Learning
Kenta Niwa
Abstract

Instead of controlling individual systems (Internet of Things (IoT) devices, smartphones, servers, etc.),
which is the current mainstream, NTT Communication Science Laboratories aims to coordinate andcontrol an overall system consisting of a set of IoT devices via digital twins. I report on the latest research
projects regarding optimal coordination of overall IoT devices using collective intelligence in digital
twins, i.e., (i) traffic coordination of autonomous vehicles and (ii) federated learning on datacenter networks.
Keywords: digital twin computing, traffic coordination, distributed learning

1. Introduction
It has become commonplace to use advanced
machine learning technology via e.g., voice commands on smartphones. The future of the machinelearning field will be learning collective intelligencefor efficient use of individual systems (Internet ofThings (IoT) devices, smartphones, servers, etc.). I
believe that a possible innovation in this field will be
optimal coordination and control of an overall system
consisting of many IoT devices, e.g., autonomousvehicles on traffic networks, servers on datacenter
networks, and power plants on energy networks. In
this article, I introduce two research projects regarding collective intelligence learning via digital twins.
2. Signal-free mobility project
Future information and communication technology
is expected to enable an advanced mobility society inwhich people, vehicles, and infrastructure cooperatewith each other to provide further safety and efficiency. In the Innovative Optical and Wireless NetNTT Technical Review

Vol. 20 No. 10 Oct. 2022

work (IOWN), signal-free mobility is presented as aconcept of this advanced mobility society. As shownin Fig. 1, vehicles autonomously travel streets withno traffic signals while communicating with eachother to shorten the time to reach destinations without
collisions. As the first step in achieving signal-freemobility, NTT Communication Science Laboratories
is tackling the problem of traffic coordination in a
distributed manner [1].
Signal-free mobility aims to predict optimal vehicle
states to shorten travel time without collisions via
feedback between digital twins and real vehicles. To
achieve optimal coordination and control of a huge
number of vehicles in real time, state transition
should be distributed, i.e., by alternatingly repeating
computation in each vehicle and communication
among vehicles. Therefore, the main research topic is
to formulate learnable digital twins to predict the
optimal states of a complex overall system in a distributed manner. Digital twins can be modeled usinga graph composed of vehicles (nodes shown as yellow vertices) and their connections (edges shown ingreen connection lines), as shown in Fig. 2. Although61

Feature Articles

Signal-free mobility (by IOWN)
Autonomous vehicles travel streets with no traffic signals while
communicating with each other to shorten the time to reach destinations
without collisions.

Current traffic coordination using traffic signals

Extracted from a video associated with IOWN announced in December 2019.
(“Mobility by IOWN,” NTT official channel, https://www.youtube.com/watch?v=4fo_kEYrY6E)

Fig. 1. Concept of signal-free mobility.

Overall system: cyber-physical system
Virtual world

Digital twins

State prediction into near future
Complex traffic situations that change from time to time
can be represented using graphical model.

Feedback loop
Real world

Control real-world systems(e.g., vehicles) in real time
Real-world systems

Fig. 2. G raphical modeling of digital twins for traffic coordination, which alternatingly repeats state prediction into the near
future and controls real vehicles in real time.

overall traffic is a very complex system that changes
from time to time, this graphical model can represent
traffic as a combination of simple components, i.e.,
local state prediction of each vehicle and communication among neighboring vehicles.
Computation procedures on digital twins for traffic
coordination is illustrated in Fig. 3. In forward propagation (Fig. 3(a), left to right), time evolution of both
optimal state transition on digital twins and feedback
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

control of real vehicles is illustrated. This is composed of alternating the repeating of multiple steps,i.e., observation-data collection (e.g., image sets ofsurrounding travel/road situations), calculation of
local vehicle state and repulsive force to maintainmore than a certain distance between vehicles, andinformation exchange through communications
among neighboring vehicles. To enable real-time traffic coordination, a vehicle’s state calculation can be62

Feature Articles

CoordiNet
Time

t0

t1

Solve ODE for collision-free traffic coordination by imposing
repulsive forcetKt2(a) Forward propagation (prediction/control)
Data aggregation Communication State prediction

Repulsiveforce

Repulsive
force
Time transition(state transition)

Digital twins
FeedbackloopState h (e.g., speed/position)is updated to go to destination.

Lossfunction(to increase
averagespeed)

Real-world
systems

State h (e.g., speed/position) is
constrained to maintain more thana certain distance.

ODE for forward
propagation

Control/data collection(b) Backward propagation (traffic-coordination model training)
Solve ODE for traffic-coordination model training to increase average speed

Time transition(model training)

Optimizing traffic-coordination model  to increase average speed

ODE forbackward
propagation

Fig. 3. C oordiNet that represents time-series processing for traffic coordination via digital twins. (a) Forward propagation(from left to right): Optimal states of vehicles can be predicted through feedback between digital twins and realvehicles. (b) Backward propagation (from right to left): Model parameters in state transition model are sequentially
optimized for efficient traffic coordination.

carried out in a decentralized manner. In backward
propagation (Fig. 3(b), right to left), learnable model
parameters in the state-transition model are sequentially updated for efficient traffic coordination byincreasing the average speed. These two flows (forward and backward propagations) were first expressedusing continuous ordinary differential equations(ODEs)*1, the spatially and temporally discretizationof which resulted in a special neural network architecture (CoordiNet), as shown in Fig. 3.
With CoordiNet, a signal-free mobility system canbe constructed, which is composed of prediction/
control and model-training phases, as shown in
Fig. 4. In the model-training phase, a number of traffic simulations on digital twins are executed. In thesesimulations, data collection assuming various traffic
situations is conducted by varying the number ofvehicles and their initial positions and traveling not
only on actual roads but also on those created in virtual worlds. The model parameters for state transition
are sequentially updated to increase the vehicle’s
average speed. Through training via a number of traffic simulations, the trained model is expected to berobust to unexpected road situations. Since thismodel-training phase is computationally heavy,model update is assumed to be executed in non-realNTT Technical Review

Vol. 20 No. 10 Oct. 2022

time (after several hours to one day). The prediction/
control phase deploys a pre-trained model for realworld traffic coordination. In the constructed system,
each feedback loop between digital twins and realvehicles is conducted in real time (in about 0.1–0.4 s).
Some of the results from the above traffic simulations for the model-training phase are shown in
Fig. 5. It was found that the average speed increased
as the traffic simulation was repeated with CoordiNet. When the speed is normalized to set its maximum value to 1.0, the average speed was increased up
to 0.90 after training compared with 0.64 before
training using randomly initialized hyper-parameters.
This result indicates that data collection via traffic
simulations is efficient for learning a traffic-coordination model. I also examined the performance of aconventional neural network (graph attention network (GAT)*2) that does not strictly restrict vehicle*1 ODE: Continuous physical phenomena such as fluids and weather are often represented by ODEs. In this report, complex phenomena in traffic coordination are represented using an ODE
such that real-world systems and digital twins can interact witheach other.
*2 GAT: A graph neural network that updates state variables while
adaptively updating the weights between connected nodes. Notethat GAT has not been proposed for application to traffic coordination.

63

Feature Articles

Executing in parallel
Prediction/control phase

Model-training phase

Alternatingly repeat of state prediction and control of real-world vehicles
by using trained traffic-coordination model in real time (0.1-0.4-s intervals)

Learning of traffic-coordination model by repeating a number of simulations to
obtain data sets assuming various traffic situations (after several hours to 1 day)
Simulation round #1

Digital twins (state transition model)

Simulation round #R
Digital twins (state-transition model)

Feedback loop

Road maps invirtual world(not exist in real world)

Feedback loop

Feedback loop

Randomly changing
number of vehicles andinitial positions
30

40

50

Digital twins (observation model)

Real-world systems

Fig. 4. Signal-free mobility system composed of prediction/control phase (left) and model-training phase (right).

CoordiNet
Methods
CoordiNet
v.s.

Dotted line is used to showthat collisions often occurred.

(R1) GAT (w/o repulsive force, trainable)(R2) CoordiNet (w/o repulsive force, trainable)(R3) CoordiNet (with repulsive force, untrainable)(R4) Native SUMO (untrainable)(a) Changes in average normalized speed through learning with each method

Average of 15
patterns oftraffic simulations

30 vehicles

hav = 0.64

50 vehicles

hav = 0.67

Before training
Average normalized
speed (maximum up
to 1.0)hav = 0.64
hav = 0.92

hav = 0.86

After training (R = 500)
Average normalized
speed (maximum up
to 1.0)hav = 0.90
Average speed is increased about 40% compared with randomly initialization.

(b) Changes in average normalized speed through learning for a specified road setting with CoordiNet

Fig. 5. Experimental evaluation of model-training phase.

NTT Technical Review

Vol. 20 No. 10 Oct. 202264

Feature Articles

Digital twins

Real-world systems

Ultrasonic beacon(GPS alternative)

Antenna(communication)

Feedback loop
CameraGPU(local computation)

Motors(rotating wheels)

GPS: Global Positioning System

Fig. 6. Experiments on real-time traffic coordination using miniature autonomous vehicles.

states to be collision-free and a traffic simulator(Simulation of Urban MObility (SUMO)*3) for comparison. Using the GAT resulted in collisions fromthe start of the learning rounds, and the average speed
could not be stably increased. When using SUMO,
however, no collisions occurred, but vehicles would
frequently stop in front of intersections, i.e., the average speed could not be increased.
To execute the prediction/control phase in real timeusing a trained traffic-coordination model (Fig. 4), asystem to control miniature autonomous vehicles wasconstructed, as shown in Fig. 6. For each miniatureautonomous vehicle, a pair of ultrasonic beacons formeasuring position, graphics processing unit (GPU)for local computation, Wi-Fi communication module,
and pair of motors to rotate wheels were implemented. In this system, each vehicle communicated withneighboring vehicles via a server and Wi-Fi, andvehicle states were predicted on the digital twins toshorten average travel time without collisions.
Around 10–20 vehicles were controllable in real time(about 0.1–0.4-s intervals). Using this system, I
experimentally confirmed that the vehicles could
travel without collisions as the state predicted by the
traffic-coordination digital twins.

ner.
The asynchronous decentralized federated learning
project aims to train model parameters (e.g., in neuralnetworks) under a massive network graph obeying alarge number of nodes and edges. A pioneering study
proposed FedAvg [2], which exchanges model
parameters and averages them between connected
nodes to make a consensus with each other. In contrast, the Edge-Consensus Learning (ECL) algorithm
I proposed and its extensions [3, 4, 5] (i) make anarbitrary graph topology available, (ii) are robust to
statistical data bias among nodes by imposing constraints on model consensus, and (iii) enable asynchronous decentralized communications, as shown in
Fig. 7.
4. Future plans
I will investigate the mathematical foundation forlearning collective intelligence and increase the number of application examples. My aim is to contributeto the early adoption of Digital Twin Computing into
society through optimal control of huge systems.

3. Asynchronous decentralized federated
learning projectIn the signal-free mobility project, the system isimplemented to learn a traffic-coordination model by
aggregating data sets obtained via traffic simulators
on a single server. However, its collective intelligencelearning phase may be shifted to be a distributed manNTT Technical Review

Vol. 20 No. 10 Oct. 2022

*3 SUMO: An open-source traffic simulator used worldwide by researchers in the intelligent transport systems field. It can be
downloaded at https://www.eclipse.org/sumo/.

65

Feature Articles

1. Makes arbitrary network topology available

Data
set #1

2. Enables data subsets for each node to be statistically biased

Data
set #2

3. Enables asynchronous communication

Data
set #3
Node #3

Node #1

Node #2

Alternatingly repeat following steps: Local node computation to update variables: Asynchronous communication between connected nodes toexchange latent variables
Update timeround1 (K=8)

Proposed algorithm (ECL-ISVR)

round2 (K=8)

Node #1
Update variables foreach local node

Node #2
Node #3

Exchange variables
via communication(data do not leak out.)

Node #N

Fig. 7. Problem settings in asynchronous decentralized federated learning.

References
[1]

[2]

[3]

K. Niwa, “Distributed Traffic Coordination without Traffic Signals,”
NTT Communication Science Laboratories Open House 2022 (Exhibition #1).
https://www.kecl.ntt.co.jp/openhouse/2022/exhibition_01_en.html
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas,“Communication-efficient Learning of Deep Networks from Decentralized Data,” Proc. of the 20th International Conference on Artificial
Intelligence and Statistics (AISTATS 2017), pp. 1273–1282, 2017.
K. Niwa, N. Harada, G. Zhang, and B. Kleijn, “Edge-consensus

[4]

[5]

Learning: Deep Learning on P2P Networks with Nonhomogeneous
Data,” Proc. of the 26th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining (KDD 2020), pp. 668–678,
2020.
K. Niwa, G. Zhang, B. Kleijn, N. Harada, H. Sawada, and A. Fujino,“Asynchronous Decentralized Optimization with Implicit Stochastic
Variance Reduction,” Proc. of the 38th International Conference on
Machine Learning (ICML 2021), 2021.
G. Zhang, K. Niwa, and B. Kleijn, “Revisiting the Primal-dual
Method of Multipliers for Optimisation over Centralised Networks,”
IEEE Transactions on Signal and Information Processing over Networks, Vol. 8, pp. 228–243, 2022.

Kenta Niwa

Distinguished Researcher, Learning and Intelligent Systems Research Group, Innovative
Communication Laboratories, NTT Communication Science Laboratories.
He received his B.E., M.E., and Ph.D. in information science from Nagoya University in 2006,
2008, and 2014. Since joining NTT in 2008, he
had been engaged in research on microphonearray signal processing. From 2017 to 2018, he
was a visiting researcher at Victoria University of
Wellington and involved with research on distributed machine learning and mathematical optimization. He is now with NTT Communication
Science Laboratories. He was awarded the
Awaya Prize by the Acoustical Society of Japan
in 2010. He is a member of IEEE (Institute ofElectrical and Electronics Engineers).

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

66

Feature Articles: Adapting to the Changing Present
and Creating a Sustainable Future

Number Theory and Quantum
Physics Based on Symmetry—
Themes from Quantum Optics
Masato Wakayama
Abstract

The connection between mathematics and physics has produced many successes over the centuries.
Modern mathematics is used, for example, in cosmology, high energy physics, and condensed mattertheory. However, the connection between the theoretical study of quantum optics and modern mathematics, which is also necessary for the construction of quantum computers, is not always deep. In this article,
I introduce the quantum Rabi model, which is the most fundamental theoretical model in quantum optics,
and its relationship with modern mathematics, especially number theory, from the viewpoint of symmetry. I discuss the motivation for my research, how it has progressed, and future plans. I also introducethat mathematical conjectures arising from this study.
Keywords: non-commutative harmonic oscillator, quantum Rabi model, zeta functions

1. Mathematics and mathematical research
The purpose of mathematical research is to enrichmathematics by solving unsolved problems and discovering new problems, as well as by developing newmathematical visions and theories that can be applied
to solutions of many problems. Mathematicalresearch also gives rise to applications that are notimmediately apparent. For instance, Riemannian
geometry, developed in tandem with physicists, islargely independent from applications and is puremathematics. It is now used, via Einstein’s theory ofgeneral relativity, in making GPS (Global Positioning
System) highly effective. Using the example of“research on quantum models and number theory
from the viewpoint of symmetry” I describe myresearch motivation, objectives and results, andfuture goals based on them. The models mentioned inthis article are the non-commutative harmonic oscillator (NCHO) [1], quantum Rabi model (QRM), andasymmetric QRM (AQRM) [2, 3].
Mathematics is, in part, an endeavor to understand,in a unified manner, facts that appear to be far apart,which sometimes takes quite long time. For example,
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

there is the application of the discovery in ancient
Greece of the infinity of prime numbers and uniqueness of prime factorization to today’s public key
cryptography. It took more than 2500 years to understand that mathematical discoveries can be used tomaintain secrecy. Also, the structure of the universe isgradually being revealed through non-Euclidean
geometry and group theory, which arose out of thehuman mind. We now know that economic phenomena and the movement of pollen can be understood in
the same way since they are due to the mathematicaldescription of Brownian motion, i.e., the development of stochastic analysis. In fact, there is a special
value in the clarification of unquestionable affinities
between subjects that have long been believed to beunrelated to each other. According to Henri Poincaré,mathematics is the art of identification. As Shin-Ichiro Tomonaga*1 wrote: “What has worked well is
experiment, demonstration/proof, and abstraction
through mathematics. Abstraction makes it very
*1 Shin-Ichiro Tomonaga was a Japanese physicist, influential in thedevelopment of quantum electrodynamics, work for which he
was jointly awarded the Nobel Prize in Physics in 1965 alongwith Richard Feynman and Julian Schwinger.

67

Feature Articles

universal. These are the great strengths of physics.”
Mathematical theorems, however, are not invented
but discovered. This becomes clear from Kunihiko
Kodaira’s view*2 that mathematical research is, using
paper and pencil, to dig out mathematical truths thatdo exist independent from human minds but are hidden.
The symmetry in the title refers to the actions ofgroups. We may recall the congruence transformationgroups of Euclidean planes and spaces and the crystallographic groups familiar from physics and chemistry. Group theory was founded by Évariste Galois,
who died in a duel at the age of 20. Before Galois,
there were formulas for roots of quadratic, cubic, andquartic equations, and there were efforts to find thosefor quintic equations. However, what Galois did wasto consider the entirety of the substitutions of theroots of an equation (the permutation group) andfound from its structure that there is in general no root
formula that uses the usual algebraic operations(addition, subtraction, multiplication, division) andapplication of radicals (square roots, cube roots, etc.)
to coefficients of a polynomial when the equation isof fifth degree or higher*3. This is a major paradigm
shift. What is the structure? It is related to the strengthof the noncommutativity of the group. There is a large
difference between putting on a suit after putting on ashirt and putting on a shirt after putting on a suit. I
hope this gives an idea of non-commutative operations (actions).
2. NCHO
Quantum harmonic oscillators are fundamental inquantum theory. Their energies (eigenvalues) are
given as half-integers in the standard normalization,and the corresponding eigenstates are Hermitianfunctions, i.e., given essentially by the Hermite polyx2

nomials times e− 2 . A clean description of this can beobtained from the representation theory*4 of thethree-dimensional Lie algebra sl2(R) consisting of 2
× 2 matrices with trace 0. The sl2(R) captures theinfinitesimal action of the Lie group SL2(R) formedby all real 2 × 2 matrices with determinant 1. Liegroup theory originated from the work of Sophus Lie,
who was impressed by the Galois theory of algebraic
equations and wanted to construct a sort of Galois
theory for algebraic differential equations. Lie groups
are groups as the name implies and geometric objects
called manifolds. In the geometric context, sl2(R) is
considered as the tangent space of SL2(R) at the idenNTT Technical Review

Vol. 20 No. 10 Oct. 2022

tity element of the group.
The Hermite functions form a basis of the Hilbert
space L2(R) of square integrable functions on the realline. Many special functions, such as Hermite functions, Bessel functions, and Jacobi polynomials, areobtained by specializing one of the parameters a, b,
or c of the Gaussian hypergeometric function F(a, b,c; x) to integer values, etc. The differential equation(Gaussian ordinary differential equation (ODE)) satisfied by F(a, b, c; x) has three regular singular points(e.g., 0, 1, ∞). The family of Gaussian hypergeometric functions is indispensable to the number theory of
elliptic curves, which has been the flower of mathematics since the 19th century. In representation theory, however, special functions derived from Gaussian
hypergeometric functions and its multivariable versions can be understood essentially as matrix elements of representations of Lie groups. I will now
turn our attention to the following.
(1)	The eigenvalues of (quantum) harmonic
oscillators H = a†a + 1 , where a and a† are2*2 Kunihiko Kodaira was the first Japanese recipient of the Fields
Medal. The K3 surface, an important algebraic surface, wasnamed by Andre Weil after three algebraic geometers (Kummer,
Kaehler, Kodaira) and the then unexplored mountain K2. In thesixth night of Soseki Natsume’s short story “Ten Nights ofDreams,” there is a description of Unkei (a Japanese sculptor during the Kamakura period; 1150–1223) carving a statue of Nioh(the guardian gods of a temple gate) at the gate of Gokokuji Temple. One of the observers caught sight of Unkei and said, “He is
not carving, he is just digging out the Nioh,” which was originally buried in the wood. This is what Kodaira used to explain.
Mathematics, which guarantees the reproducibility essential toscience through rigorous proofs, is not a natural science that
deals with physical nature, but rather a science that works toward
the elucidation of mathematical nature.
*3 This is the beginning of Galois theory. Many mathematicians
have become mathematicians because they were exposed to Galois
theory. Although we now call it a “solution formula” for quadratic
equations in Japan, it used to be called a “root formula,” since a
quadratic equation is one of various equations. However, I would
like to use “root” with a special meaning because it is indeed aroot which has broadened the world of mathematics.
*4 Representation theory is said to be the study of symmetry. It is at
the intersection of algebra, geometry, and analysis, and is indispensable in mathematics that focuses on treating symmetry. In
representation theory, each group element is represented by a linear transformation acting on a certain vector space (generally infinite dimensional) and the group product corresponds to composition of these transformations. We use the Weil representation(sometimes called the oscillator representation), the representation of sl2 (R) given by the multiplication 1 x2, the differential22
operator − 1 d 2 , and Euler’s order operator x d + 1 . (For an
2 dx
dx 2
introduction to the Weil representation, see R. Howe and E-C.
Tan, “Non-Abelian Harmonic Analysis: Applications of SL(2, R),”
Springer, 1992.)

68

Feature Articles

respectively the creation and annihilation
operators subjecting [a†, a] = 1, are half-integers. Therefore, the spectral zeta function*5 ofthe harmonic oscillator determined from theeigenvalues essentially coincides with the
Riemann zeta function ζ(s)*6.
(2)	The symmetry of SL2(R) remains in the worldof Gaussian hypergeometric functions. For
instance, the matrix elements of representations of SL2(R) and, almost equivalently, thesolution of the SL2(R)-invariant differential
equation all can be expressed essentially by
Gaussian hypergeometric functions.
The group SL2(R) is behind (1) and (2) above. To
explore the wider world, we need to weaken the symmetry. The only noncommutative property of harmonic oscillators is the canonical commutation relation (CCR) controlled by sl2(R)*7. The NCHO*8,
which defines a system of ODEs, with two parameters α and β, adds matrix noncommutativity. The
name comes from this noncommutativity. Of course,
it would not be rational to define it in an arbitrary
manner. From expected physical applications andcorresponding mathematically reasonable consideration, we thought that it should be at least requiredthat the Hamiltonian Q of the NCHO have only real
discrete eigenvalues and that it be kept a sufficiently
weak symmetry to allow the harmonic oscillator to be
recovered when α = β. The former requires that Q be
a positive Hermitian operator (i.e., self-adjoint operator). The latter is that the spectral zeta function ζQ(s)of Q be a good generalization of ζ(s) though neither
a functional equation nor Euler product may be
expected [4].
The convergence region of the defining series ofζ(s) is only in the half-plane, while ζ(s) has analytic
continuation to the whole plane. The RiemannHypothesis*9 asserts that the real part of an imaginary
number s with ζ(s) = 0 is always 1 . There are two
2
analogues of the Riemann Hypothesis at the top (andamong the best-known achievements) of 20th century
mathematics. One is the Weil Conjecture for the congruent zeta function and the other is the Selberg zetafunction for the prime geodesic distribution. Bothconjectures were settled by the fact that each zetafunction has a determinant expression in terms ofsuitable Hermitian operators*10. The study of a possible determinantal expression of ζ(s) is also important in the attack on the original Riemann Hypothesis.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

3. QRM and NCHO
Isidor Isaac Rabi described the interaction of classical light with atomic quantum states in 1936. In
1963, Edwin Jaynes and Fred Cummings quantizedlight as well and defined the QRM and its rotating
wave approximation, now known as the JaynesCummings model. However, while the integrabilityof the QRM was unknown, the Jaynes-Cummings
model was useful because of the existence of continuous invariants, making the theoretical treatment easy,and suitability to experiments if the light-matter coupling is very weak, roughly a fraction 10−6 of themode frequency*11.
However, with the possibility of developing a quantum computer on circuit quantum electrodynamics(QED) platforms, the weak coupling approximation
is no longer justified, and one must use the QRM in
the ultra-strong coupling regime. In 2011, Daniel
Braak [2] demonstrated the integrability of the QRM,
and research on the periphery of the QRM has
advanced rapidly since. In fact, the experimental
measurement results [5] using superconducting*5 The spectral zeta function of an operator with positive eigenvalues λ1≦λ2≦λ3≦ …, where the multiplicity of the eigenvalue is
uniformly bounded, is defined by the Dirichlet series λ1−s +λ2−s+λ3−s+....
*6 ζ(s): = 1+2−s +3−s +4−s +5−s + … = (1–2−s ) −1×(1–3−s ) −1×(1–5−s ) −1
×(1–7−s)−1×(1–11−s)−1 × … (Euler’s product) for a complex number s, the real part of which is greater than 1. The spectral zetafunction is in fact identified with (2s − 1)ζ(s).
2*7 Define h = a†a + 1 , e+ = 1 a† and e− = − 1 a2, then the triplet {h,222e+, e−} forms the Lie algebra sl2(R), that is, the commutation relations [h, e±] = ±2e± and [e+, e−] = h hold. Here h = 2H + 1 for
2
the Hamiltonian of harmonic oscillator H = 1 a†a.
2
α 0
*8 The Hamiltonian Q of the NCHO is of the form Q :=
0 β
1 d2
1 2
0 −1
d + 1 , where α and β are posix−++x
2 dx2 2dx 2
1 0tive and αβ > 1. The condition on α and β guarantees that Q has
only positive and discrete eigenvalues with uniformly bounded
multiplicity.
*9 It is still unsolved more than 160 years after being stated by Riemann (1859). This conjecture, which has been challenged bymany geniuses, is equivalent to the ultimate distribution ofprimes, as shown in 1901 by Helge von Koch. Koch is famous
for the Koch snowflake/curve, after whom it is named.
*10 The congruence determined for smooth algebraic manifolds andschemes over finite fields was positively solved in the case of
zeta functions by the determinantal expression of what is called
the Frobenius operator, and in the case of Selberg zeta functions
by the Laplacian, the (unique up to a constant) invariant differential operator on the complex upper half-plane.
*11 The Nobel Prize in Physics 2012 awarded to Serge Haroche andDavid J. Wineland for their groundbreaking experimental method
for the measurement and manipulation of quantum systems
seems to be at the center of this trend.

(

) (

)(

(

)

)

69

Feature Articles

The Weil Representation
4 regular singular points
0,1, t(= αβ), ∞Number theoretic structure behind NCHONCHO is a covering model of QRM.

Apéry-like numbers(and their generating functions)

Special values of the spectral
zeta function

Congruence relation(with higher prime powers)

Fuchsian ordinary
differential equation

Arithmetic geometry
Modularity (can be written in terms ofautomorphic form and integral)

Courtesy of APS/Alan Stonebraker (2011)

Light-matter interaction

MahlermeasureWeighted Cayley graph
Discrete Dynamical
Systems

Fig. 1. NCHO & QRM.

artificial atoms were consistent with the theory of the
QRM and its generalization, the AQRM (see also
[6]).
The NCHO has expanded the world from Gauss’s
as expected. In fact, the eigenvalue problem of the
NCHO was found to be equivalent to the existence ofholomorphic solution of the Heun ODE*12 in thecomplex region containing 0, 1, and not including αβ.
This fact was discovered by Hiroyuki Ochiai [7] andfully understood 15 years later [8]. The (A)QRM also
has a Heun picture [2]. However, unlike the NCHO,
it is a confluent Heun differential equation. Nevertheless, by merging the two singularities as αβ → ∞, a
confluent Heun picture of the QRM can be obtained
from the Heun picture of the NCHO [8]. In otherwords, the NCHO can be considered as a coveringmodel of the QRM*13 (Fig. 1, the NCHO is a coveringmodel of the QRM).
Quantization is discretization and passing from acommutative world to a non-commutative world. The
former is the original arena of number theory. In fact,
the special values of ζQ(s) in the NCHO shows richnumber theory as I explain below. For instance, as tothe special values of the spectral zeta function of theharmonic oscillator, essentially given by ζ(s), there is
the old Basel problem to determine the value of the

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

infinite series 1 + 1 + 1 + 1 + 1 + …*14, which was
4 9 16 25
solved by Euler. The even integer values of ζ(s) are
generally expressed in terms of powers of π and rational numbers. However, the value and properties ofζ(s) at odd points remained unknown for more than
300 years until Roger Apéry’s proof that ζ(3) is irrational in 1978. The best result on irrationality is still
the theorem that at least one of ζ(5), ζ(7), ζ(9), orζ(11) is an irrational number. However, the researchon special values is a deep problem in modern mathematics, such as the geometric conjecture on special
values of various zeta functions by Pierre Deligne*15
who solved the Weil Conjecture using Grothendieck’s
arithmetic geometry, and the Birch-Swinnerton-Dyer*12 The Heun equation is a second-order ODE of Fuchsian type withfour fixed singular points, e.g., (0, 1, t, ∞). Unlike Gaussian hypergeometric functions, there are aspects of its mathematicalstructure that need to be clarified, but they appear in the description of various physical phenomena. The Gaussian hypergeometric equation can be considered a degenerate case of the Heun
equation. This is the first meaning for expanding the world fromGauss’s.
*13 This is also true for the AQRM by adding an appropriate shifted(biased) term to the NCHO.
*14 In terms of the special value of ζ (s), 1+ 1 + 1 + 1 + 1 + … =4 9 16 25
2ζ (2) = π . The appearance of π was a great surprise.
6*15 Fields Medal 1978.

70

Feature Articles

conjecture on special values at the central point of
L-functions (relatives of zeta) defined from elliptic
curves and its generalization by Bloch, Beilinson, and
Kato.
Apéry used a sequence of rational numbers, now
called the Apéry numbers, to prove the impossibility
that ζ(3) is rational. Frits Beukers took up the challenge of finding out why and further proving irrationality and developed a brilliant study of elliptic
curves, K3 surfaces, and automorphic forms behindthe proof of the irrationality [9]. As a result, heshowed remarkable congruence relations of the
Apéry numbers, which have become the subject of
research in algebraic number theory. Surprisingly, thespecial values ζQ(2) and ζQ(3) for the NCHO also
yield sequences of numbers with properties like thoseof the Apéry numbers [10, 11, 12]. Elliptic curves andautomorphic forms underlie these properties, andwhen s = 4, even automorphic integrals*16, which
generalize the notion of automorphic forms, appear[10] via certain nearly holomorphic automorphic
forms initially studied by Goro Shimura*17. The next
development will be the study of a wider range/classof automorphic forms (also related to the Langlands
program, which is extremely important in mathematics [13]).
4. Spectra of the AQRM and hidden symmetries
The importance of the partition function (weighted
sum of states) in statistical mechanics is that it
encodes the time evolution of the system. Knowing
the exact behavior of a quantum system during a long
period is of utmost importance to any task in quantum
computing. Purely numerical approaches are severely
limited due to the accumulation of rounding errors in
the long-time limit. Therefore, it is necessary toderive analytical formulas for its heat kernel and partition function. In fact, the partition function andspectral zeta function have a one-to-one correspondence through the Mellin integral transform. However, obtaining these analytical formulas was considered extremely difficult. Physicists use a highly intuitive form of expression called Feynman path integral
to carry out various asymptotic or approximate (aswell as precise) calculations to gain a better understanding of the physical system of interest. The translation invariant measure and integral on the space ofpaths cannot be treated mathematically in a rigorous
manner, except in some special cases. Of course,
various studies are still ongoing to understand it. I,
together with Reyes-Bustos, have attempted to derive
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

a heat kernel for the (A)QRM*18. The method, which
has been successfully applied [14], is based on theaction of the symmetric group 𝔖m of the mth order on*19, andZ2m(m = 1,2,...), the Fourier transform on Zm
2graph theory, where Z2 is Z/2Z or the field of two elements F2 = {0,1}. The resulting analytic formula ispractically given by the series over non-negative integers m for integrals over m-simplex (an m-fold integral) of elementary functions given by products andquotients of exponential functions. It can also beinterpreted as the infinite sum (integral) of the orbit
integrals of 𝔖∞ over the entire equivalence class by
the infinite symmetric group 𝔖∞ of discrete paths
connecting 0 and ∞ (Fig. 2, Discrete paths and infinite symmetry groups (Geometric Interpretation)).
From a representation theoretic point of view, theaction of 𝔖∞ on Z2∞ corresponds to a decomposition
into the direct sum of irreducible representations(Fig. 3 (Algebraic Interpretation)). The formulas
obtained are similar to formulas for partition functions obtained for the spin-boson model or the Kondo
model (describing magnetic impurities in non-magnetic host metals, named after Jun Kondo, 1930–2022), a formally exact description of the matrix
element of the heat kernel, and to approximate partition functions for various physical systems. For ageneral model, the (possible) existence of an infinitedimensional algebraic system acting on the noncountable paths of the Feynman integrals and theacquisition of discrete paths as their equivalenceanalogue should also be pursued (Fig. 4, Conjectureon the paths of the Feynman path integral).
From the analytical formula for the heat kernel, thepartition function is obtained, and the contour integral expression of the spectral zeta function isgained*20. Thus, the analytic connection to the wholeplane is obtained, the asymptotic evaluation of thenumber of eigenvalues (Weyl’s law) follows, and we
can start the special value study [15]. What is thenumber theory and geometric structure of the QRMs?
*16 Also called the Eichler form.
*17 This fact means that the second reason we can expand the worldfrom Gauss’s. The notion of automorphic integrals (sometimes
called Eichler’s forms) appeared in [10] is a further reasonable
extension of the standard one.
*18 The Trotter-Kato product formula is the starting point for the heat
kernel derivation; it is useful to derive the Feynman path integral
formally, or even considered as the rigorous description of the
Feynman path integral.
*19 It is a part of the action of the oscillator representation of SL2(F2)n.
This representation is used for the construction of quantum error
correcting codes such as Clifford cord.
*20 The contour integral is a path starting from the infinity point ∞,circling around the origin 0, and returning to ∞.

71

Feature Articles

: = {ℴ ∈ ℤ2∞: |ℴ| = } = ∞ [1,1,…,1](length | • | = ∞-invariant)(- orbit)∞([1,1,…,1] = (1,1,…,1,0,0,……) ∈ ℤ2∞)(the orbital decomposition of ℤ2∞)ℤ2∞ = ∐∞=0 ∞QRM () =∑ (Δ)
=02

exp[   ­    ­   ­ ,,,,  ]  (Orbital integral can be regarded as an integral over  − simplex)[equivalence class of paths]

Fig. 2. Heat Kernel Formula – Geometrical Interpretation.

Irreducible decomposition of ℤ2∞  ℎ  

∞

:

∞

ℤ2∞ = ⊕∞=0 Ind ∞ 1∏: Young subgroup∏(Young diagram) ∏ =∞corresponding to the partition

(

...


)

... .

,
∞

There is an idea (due to Ludvig Faddeev) that the construction of any irreducible representation
can be obtained by Feynman path integrals for a Lie group via co-adjoint orbits.
This is reminiscent of our analytical formula of the heat kernel.

Fig. 3. Heat Kernel Formula – Algebraic (Group Theoretic) Interpretation.

This is one of the most important questions in thisarea.
The AQRM is of major importance in circuit QED
implementations of the QRM, thus for applications toquantum computing. Unlike the QRM, it has no visible Z2-symmetry (parity), if the bias is not zero.
Nevertheless, it shows characteristic spectral degeneracies at half-integer values of the bias parameter. In
fact, the AQRM is degenerate if and only if the bias
parameter (of the bias operator) of the Hamiltonian*21
is a half-integer ℓ/2 (ℓ ∈ Z) [3, 16]. In physics, however, degeneracy is considered to indicate the existence of a symmetry [17]. Unlike classical mechanics, the concept of integrability is not clear in quantum mechanics [18]. However, the analogy withclassical systems has led to the search for operators
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

that commute with HAQRM but are not functions of
HAQRM, indicating the presence of an additional conserved quantity [19]. Inspired by this study, we [20]conducted a thorough analysis to completely determine the operators that are commuting with the
HAQRM. Interestingly, the ring*22 of operators commuting with HAQRM, has a unique natural generator Jℓ*21 The AQRM Hamiltonian has the form H AQRM = a †a [photon
field] + ∆ σz [two-level atom] + g(a + a†) σx [interaction term] +2
ϵσx [bias term (given by a real number)], where σ* is Pauli matrix, Δ is the energy difference of the two level atom and g thecoupling (interaction) constant. We notice that the bias term
given by ϵ ∈R of HQRM is zero. Note that after its proposal in [2],the AQRM (as it is now called) was called the “generalized”
QRM, “biased” QRM, “driven” QRM, etc., but after [3], “asymmetric” QRM became established.

72

Feature Articles

A, B: Bounded operators from below(No suitable additive measure)

Representation Theory ofSL2( 2)nClifford group,
quantum error
correcting code
constructionnℤ2∞: = limℤ→n 2
∞: infinite symmetric group

Graph Theory

Conjecture{Feynman Path}/~ ≈ {Discrete Path}(equivalence classes by some infinite dimensional
algebraic system e.g. group)

Fig. 4. Product Formulas of Trotter-Kato

up to a constant multiple. Moreover, the square of Jℓis expressed as Jℓ2 = pℓ (HAQRM, g, Δ), a polynomial
in HAQRM, where pℓ is an ℓth degree polynomial. The
equation y2 = pℓ (x; g, Δ) defines a hyperelliptic curve[21].
On the other hand, the degenerate eigenvalues of
the AQRM are of the form N ± ℓ + g2 with positive2
integer N. Fixing N and ℓ, (g2, Δ2) is obtained as zeros(−ℓ/2) 2(g , Δ 2) andof the constraint polynomial P N+ℓ

P N(ℓ/2)(g2, Δ2) [12]. In fact, P N(ℓ/2)(g2, Δ2) divides(−ℓ/2) 2(g , Δ 2 ), and the quotient polynomial is
P N+ℓAℓN(g2, Δ2) (>0) [16]. It should also be noted that the
AQRM degeneracies are, geometrically, conicalintersection points in the three-dimensional spectral
graph (Left of Fig. 5).
From experimental calculations, we know thatN 2
Aℓ (g , Δ2) = pℓ N − 1 + g2, g, Δ (ℓ ≤ 6). If this holds
2
in general, the expectation “hidden symmetry behindthe degeneracy” becomes explainable concretely provided we may think of the existence of a non-trivial
operator commuting with the Hamiltonian (which isin fact an involution modulo the Hamiltonian) as evidence of quantum integrability*23 (Fig. 5 and Fig. 6).
There is, however, no way to prove it, and it remains(

NTT Technical Review)

Vol. 20 No. 10 Oct. 2022

Heat Kernel, Propagator.

a conjecture [21]. However, this conjecture is very
similar in form to a certain theorem*24 in the Diophantine geometry*25 [22] in its form. Therefore, aclue to the proof might be found there. We also expectthat the AQRM has a finite number of degenerate
eigenvalues for fixed parameters g and Δ. Although it
is still unresolved, we can find some formal similarities with the Mordell Conjecture*26 in this case.

*22 It is an algebraic system closed in addition and multiplication except for division, such as the whole of integers (integer ring) andthe whole of polynomials (polynomial ring).
*23 We can define a new mathematical model that we call the shifted
NCHO. This model gives the covering model of the AQRM bylooking at each Heun ODE picture for the shifted NCHO andAQRM, respectively. Since the model keeps the apparent parity
symmetry at the Hamiltonian level, the hidden symmetry of the
AQRM may also be explained as the inheritance of this parity
symmetry.
*24 This theorem is closely related to the well-known Vojta Conjecture. The Vojta Conjecture is also a higher dimensional version of
the Mordell Conjecture and closely related to the ABC Conjecture.
*25 A fundamental theme in number theory concerns the study of integer and rational solutions to Diophantine equations (polynomial
equations). This topic originated at least 3,700 years ago as documented in Babylonian clay tablets. There are interesting interactions between Diophantine Geometry and several fields such as
representation theory, algebraic geometry, topology, complex
analysis, and mathematical logic, to mention a few, which havebeen found to be quite fruitful.

73

Feature Articles

What physicists believe(long unknown)

Eigenvalue degeneracy

Eigenvalues are
essentially integers
and parametrized.

Hidden symmetry

Concrete description

Description byhyperelliptic surfaces

Constraint polynomial
commutativityℓ and quadratic formula
Quotient ofℓConstraint Polynomials
defined by the generators of
expectation the commutative operators(Polynomials)

Symmetry hidden in AQRM degeneracy ( = ℓ ∈ ℤ)
2
Eigenvalue (integer) contractions: integer
divisors of constrained polynomials

resemble
Polynomial Integral and Integer Store Reduction in
Diophantine Geometry(Deeply related to Vojta forecast)P. Corvaja and U. Zannier, Adv. Math., 225 (2010)

Constraint Polynomials ℓ(, ∆) = 0⇨
Juddian exceptional eigenvalue N − 2 + ℓ exists
2

Using the idea of adiabatic approximation using
Constraint polynomials, an approximation of thegeneral spectral curve can be obtained.
Zi-Min Li et al, J. Phys. A: Math. Theor.,
54, 405201 (2021)

Assuming the expectation, the first ℓ spectral
curves of ℓ are very well approximated by ℓ

Gray: Energy and spectral curves
Red: ℓ = 0Constraint polynomials can be said that thedegenerate eigenvalues “know” the entire system.

Fig. 5. Clarification of the underlying fundamental reason is expected.

Main conjectureℓ ∈ ℤ≥0,  ≥ ℓ ⇒Δ = 4, ℓ = 3

The mainconjecture istrue ⇒
Grey: spectral curve on(g,E)-plane
Red: curve ℓ(, , Δ) = 0
Dotted line: the baseline curve  =  − ℓ −2( = 0,1,2)
2: exceptional eigenvalue

Fig. 6. The degenerate eigenvalues may “know” the entire spectrum of the system.

5. Conclusion
Let us discuss a few phrases that characterize mathematics. First, “All things are numbers” by Pythagoras.
This is reminiscent of the big data and artificial intelligence (AI) that dominate modern society. The
genius Leonardo da Vinci said, “Engineering is theparadise of the mathematical sciences. After all, it ishere that the fruits of mathematics are borne.” Galileo
Galilei’s “The universe is written in mathematical
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

language” is well known. Wigner’s “The unreasonable effectiveness of mathematics in the natural sciences,”*27 is a brilliant point.
Although mathematical research is like musicalcomposition in some respects, it has been difficult tofind a mechanism to make mathematical papers play
*26 The conjecture that a curve with a number of holes (genus) greater than 1 defined on a rational number field would have only a finite number of rational points, proved by Gerd Faltings in 1983.

74

Feature Articles

like music, such as reproducing a symphony from ascore. However, the use of mathematics has emerged
because of machine learning, AI, and other technologies. Although the motivations for research arediverse, including pure mathematics driven solely by
curiosity, it now seems as if an opportunity has
arrived for basic mathematics and applied mathematics to play a brilliant concerto. I close this article withthe hope that a new “mathematical music” (in Japanese, we may express it as(su-gaku) meaning
enjoining mathematics and is a homonym with
(su-gaku) meaning the study of numbers, i.e., mathematics)*28.
References
[1]
[2]
[3]
[4]
[5]

[6]
[7]
[8]

A. Parmeggiani and M. Wakayama, “Oscillator Representations andSystems of Ordinary Differential Equations,” PNAS, Vol. 98, pp.
26–30, 2001.
D. Braak, “Integrability of the Rabi Model,” Phys. Rev. Lett., Vol.
107, 100401, 2011.
M. Wakayama, “Symmetry of Asymmetric Quantum Rabi Models,” J.
Phys. A: Math. Theo., Vol. 50, pp. 174001–174020, 2017.
T. Ichinose and M. Wakayama, “Zeta Functions for the Spectrum of
the Non-commutative Harmonic Oscillators,” Comm. Math. Phys.,Vol. 258, pp. 697–739, 2005.
F. Yoshihara, T. Fuse, Z. Ao, S. Ashhab, K. Kakuyanagi, S. Saito, T.
Aoki, K. Koshino, and K. Semba, “Inversion of Qubit Energy Levelsin Qubit-oscillator Circuits in the Deep-strong-coupling Regime,”
Phys. Rev. Lett., Vol. 120, 183601, 2018.
M. Wakayama, “Quantum Interaction and Special Values of Zeta
Functions,” Sugaku Tsushin, Math. Soc. Japan., Vol. 25, No. 4, pp.
24–52, 2021 (in Japanese).
H. Ochiai, “Non-commutative Harmonic Oscillators and Fuchsian
Ordinary Differential Operators,” Comm. Math. Phys., Vol. 217, pp.
357–373, 2001.
M. Wakayama, “Equivalence between the Eigenvalue Problem ofNon-commutative Harmonic Oscillators and Existence of Holomor-

phic Solutions of Heun Differential Equations, Eigenstates Degeneration, and the Rabi Model,” Int. Math. Res. Not., Vol. 145, pp. 759–
794, 2016.
[9] F. Beukers, “Irrationality of π2, Periods of an Elliptic Curve and Γ1(5),” Diophantine Approximations and Transcendental Numbers(Luminy, 1982), Progr. Math., Vol. 31, pp. 47–66, 1983.
[10] K. Kimoto and M. Wakayama, “Apéry-like Numbers for Non-commutative Harmonic Oscillators and Automorphic Integrals,” Ann. Inst.
Henri Poincaré D (to appear), arXiv:1905.01775, 2019.
[11] L. Long, R. Osburn, and H. Swisher, “On a Conjecture of Kimoto andWakayama,” Proc. Amer. Math. Soc., Vol. 144, No. 10, pp. 4319–
4327, 2016.
[12] J.-C. Liu, “A Generalized Supercongruence of Kimoto andWakayama,” J. Math. Anal. Appl., Vol. 467, No. 1, pp. 15–25, 2018.
[13] R. Langlands, “Letter to André Weil,” https://publications.ias.edu/rpl/
paper/43[14] C. Reyes-Bustos and M. Wakayama, “The Heat Kernel for the Quantum Rabi Model,” Adv. Theor. Math. Phys. (to appear),arXiv:1906.09597, 2020.
[15] C. Reyes-Bustos and M. Wakayama, “Heat Kernel for the Quantum
Rabi Model: II. Propagators and Spectral Determinants,” J. Phys. A:
Math. Theor., Vol. 54, pp. 115202–115231, 2021.
[16] K. Kimoto, C. Reyes-Bustos, and M. Wakayama, “Determinant
Expressions of Constraint Polynomials and the Spectrum of the
Asymmetric Quantum Rabi Model,” Int. Math. Res. Not., Vol. 2021,
No. 12, pp. 9458–9544, 2021.
[17] S. Ashhab, “Attempt to Find the Hidden Symmetry in the Asymmetric
Quantum Rabi Model,” Phys. Rev. A, Vol. 101, 023808, 2020.
[18] J. S. Caux and J. Mossel, “Remarks on the Notion of Quantum Integrability,” J. Stat. Mech., 02023, 2011.
[19] V. Mangazeev, M. Batchelor, and V. Bazhanov, “The Hidden Symmetry of the Asymmetric Quantum Rabi Model,” J. Phys. A: Math.
Theor., Vol. 54, 12LT01, 2021.
[20] C. Reyes-Bustos, D. Braak, and M. Wakayama, “Remarks on the Hidden Symmetry of the Asymmetric Quantum Rabi Model,” J. Phys. A:
Math. Theor., Vol. 54, pp. 285202–285215, 2021.
[21] C. Reyes-Bustos and M. Wakayama, “Degeneracy and Hidden Symmetry for the Asymmetric Quantum Rabi Model with Integer Bias,”
Commun. Number Theor. Phys. (to appear), arXiv:2106.08916, 2021.
[22] P. Corvaja and U. Zannier, “Integral Points, Divisibility between Values of Polynomials and Entire Curves on Surfaces,” Adv. Math., Vol.
225, pp. 1095–1118, 2010.

*27 The following is an excerpt from a lecture transcript published in1960 by Eugene Wigner (1902–1995: Nobel Prize in Physics
1963, “Discovery of Symmetry in the Theory of Nuclei and Elementary Particles”). Wigner’s remark echoes a well-known sentence by Einstein, who said, “Das ewig Unbegreifliche an der
Welt ist ihre Begreiflichkeit.” in English: “The fact that the universe is comprehensible [by mathematical means] is utterly incomprehensible.”*28 Many professions in academia, e.g., scientist, physicist, chemist,
and linguist end in “st.” However, astronomer and geometer end
in “er.” Magician, musician, and mathematician end in “cian,”and their initial letter is “m.” Perhaps there might be some influence of liberal arts education (the seven liberal arts) dating backto ancient Greece and Rome, but it is not easy to explain immediately. It is just interesting to notice.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

75

Feature Articles

Masato Wakayama

Head and Fundamental Mathematics Research
Principal of NTT Institute for Fundamental
Mathematics.
He graduated from Tokyo University of Science in 1978 and completed a doctoral course at
Hiroshima University’s Graduate School of Science in 1985 (Doctor of Science). Before becoming a research principal at NTT Institute for
Fundamental Mathematics on October 1, 2021,
he had held many other academic positions: anassociate professor at Tottori University andKyushu University, visiting fellow at the Department of Mathematics, Princeton University, visiting professor at the University of Bologna, distinguished lecturer at Indiana University (Bloomington), distinguished professor in Mathematics,
Dean of the Graduate School of Mathematics,
Kyushu University, the first director of the Institute of Mathematics for Industry, executive vicepresident of Kyushu University, and vice president and professor of Tokyo University of Science. He is also a principal fellow of CRDS
(Center for Research and Development Strategy,
Japan Science and Technology Agency), and aprofessor emeritus of Kyushu University. He
specializes in representation theory, number theory, and mathematical physics.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

76

Regular Articles

Development of a Rust Removal Tool
Using a Diffractive Optical Elementand High-power Laser and Its
Application to Asbestos-melting Tools
Sohan Kawamura and Kaori Takayanagi
Abstract

To improve the efficiency of rust removal on telecommunication towers, we developed a prototype ofa compact, lightweight rust-removal tool using a diffractive optical element and high-power laser thatweighs only 500 g. With the aim of applying this technology to asbestos removal, we experimentally
irradiated asbestos with a laser, and the results of the experiment suggest that using a laser in this manner
can melt and remove asbestos and reduce its risk as a health hazard to workers.
Keywords: infrastructure, high-power laser, diffractive optical element

1. Introduction
NTT possesses a large amount and wide variety of
infrastructure facilities that are indispensable for
implementing NTT’s concept called the Innovative
Optical Wireless Network (IOWN). To efficiently
maintain these facilities, NTT is researching anddeveloping technology that involves using highpower lasers and diffractive optical elements. In thisarticle, a rust-removal tool using a high-power laserand diffractive optical element is described, and theresults of experiments conducted to apply this technology to asbestos removal are presented.
2. Development of rust-removal tool
NTT maintains 20,000 telecommunication towers,
many of which are several decades old, so it is essential to remove rust from and repaint them to prevent
them from deteriorating. Currently, electric tools areused to remove rust; however, the space around bolts
is too tight for electric tools to remove the rust there.
Lasers have been attracting attention as a means ofremoving rust. Lasers can melt and vaporize rust
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

without contacting it, so rust can be removed from
confined spaces without damaging the base material.
However, a conventional laser system consists of amechanical-drive unit that scans a laser beam(focused by a lens) in two dimensions, and the systemis large and heavy, weighing about 2 kg, thus difficult
to use high above the ground in places such as theupper reaches of telecommunication towers.
Given the above-described circumstances, NTT
aimed to develop a small, lightweight rust-removal
tool that can be used high above the ground. We havedeveloped a prototype rust-removal tool—weighing
only 500 g—that can be held in one hand. We
achieved this by eliminating the conventional
mechanical-drive unit, which was replaced with a diffractive optical element for forming the laser beamwith a narrow, linear intensity distribution and moving the shaped laser beam by hand [1]. The prototype
laser system and examples of using it for removing
rust are shown in Fig. 1. To prevent rust forming on
steel towers, it is also important to prevent the paintfrom peeling off after the rust is removed. Accordingly, we confirmed that this technology (i.e., rustremoval tool) leaves sufficient paint adhesion to77

Regular Articles

Fig. 1. Prototype rust-removal tool (left) and example of rust removal (right) [1].

prevent rusting.
3. Application to asbestos removal
3.1 Demand for asbestos removal
Some of the facilities owned by NTT include buildings that contain asbestos, which must be removed
when the building is demolished or renovated. However, there are several issues with methods of removing asbestos. We are therefore also researching theabove technology to apply it to removing asbestos
from buildings.
Asbestos is a fibrous substance that has been produced and used in large quantities since the 1920s
because of its excellent heat and chemical resistance.
In the 1950s, however, asbestos was found to causelung cancer, and its production and use were bannedin Europe from the 1990s and in Japan in 2004 [2–5].
According to Japan’s Ministry of Land, Infrastructure, Transport and Tourism, the number of demolitions of buildings containing asbestos in Japan has
increased and is estimated to reach 100,000 buildings
around 2028. The demand for demolition work is
estimated to continue for about 30 years [6]. Thosebuildings are being demolished after asbestos-containing painted materials are removed by using power
tools or water jets.
However, power tools invariably scatter asbestos,so equipment is needed to prevent leakage of asbestos
and exposure of workers to it. If water jets are used,scattering of asbestos can be controlled, but it is difficult to remove asbestos from narrow spaces, andequipment for filtering water mixed with asbestos isessential.
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Asbestos-containing materials that have been
removed are either melted down to eliminate its fiber
structure and reused or disposed of as landfill. Owing
to the above-mentioned increase in demolition workin the future, it is a concern that there will be theshortage of asbestos-treatment facilities and landfill
sites. Asbestos-treatment technology that enables
low-cost and efficient reuse is thus required [7].
We are researching and developing technology for
reducing the risk of worker exposure to asbestos andleakage of asbestos by using high-power lasers toremove asbestos in a manner that melts it and destroys
its harmful fibrous structure. High-power lasers canmelt asbestos in a non-contact manner, so it becomes
possible to remove asbestos in narrow spaces fromwhich it cannot easily be removed with power tools
or water jets. Hereafter, an experiment in whichasbestos was melted by using a laser is described.
4. Asbestos-melting experiment
4.1 Sample preparation
Chrysotile (Mg3SiO5(OH)4) powder, i.e., the mostcommon form of asbestos, was used to confirm thatlaser melts asbestos. To observe the dust generated bylaser irradiation of asbestos-based coating materials,
coating materials collected from buildings were usedin the experiment. The coating material consists of abase coat, main coating material, and top coat. Asbestos is used as the base-coat material when mixed withmortar. The appearance and cross section of anasbestos-containing coating material taken from abuilding and used in this experiment are shown in
Fig. 2. The thickness of the base coat of the sample
78Regular Articles

Top coating material

Main material
Substrate-adjusting material
Concrete

Fig. 2. A ppearance (left) and cross-sectional view (right) of asbestos-containing coating material collected from a building[8].

Fig. 3. Electron micrographs of chrysotile (left) and laser-irradiated chrysotile (right) [8].

was about 50 µm, and the concentration of the asbestos (chrysotile) contained in the sample was 1 wt%.

amount of dust was qualitatively evaluated from theintensity of the reflected light.

4.2 Laser-irradiation experiment
A pulsed laser—with average output of 100 W, frequency of 100 kHz, and wavelength of 1.06 μm—wasused. The laser beam emitted from the light source
was focused using an fθ lens to spot diameters of 50
and 140 μm and scanned in two dimensions with agalvanometer mirror over an area of 8 × 8 mm.
To qualitatively estimate the amount of asbestos
dispersed by laser irradiation, the dust generated bylaser irradiation was captured with a high-speed camera. It was assumed that the amount of asbestos dispersed was proportional to the amount of dust generated. The sample was illuminated using a laser with awavelength of 532 nm and, simultaneously, a highpower laser with a wavelength of 1.06 μm. The lightwith the wavelength of 532 nm reflected by the dust
was captured with the high-speed camera, and the

4.3 Experimental results
Electron micrographs of the chrysotile before andafter laser irradiation are shown in Fig. 3. It is clearthat the laser irradiation destroyed the fibrous structure of the chrysotile. Analysis of the composition of
the particles observed after laser irradiation showedthat particles containing magnesium and silicon weregenerated. Therefore, it is considered that the particles were the result of the chrysotile melting thensolidifying.
The results of qualitatively estimating the amountof dust on the coating material (collected from abuilding) when it was irradiated with the pulsed laser
are plotted in Fig. 4. Images of the dust were captured
with a high-speed camera, the brightness in theimages was calculated, and the amount of dust wasestimated from the calculated brightness.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

79

Regular Articles

1.0
Beam spot diameter 50 µm
Beam spot diameter 140 µm

Normalized intensity

0.8

0.6

0.4

0.2

0.0

0

2

4

6

8

10

Irradiation time (sec)Fig. 4. Example of brightness measurement using a high-speed camera image when beam diameter is changed [8].

Since the brightness of the image changes bychanging the beam diameter, it can be assumed thatthe amount of dust also changes. This suggests thatthe dust can be suppressed by appropriately controlling the laser-irradiation conditions. An example ofone of the particles in the dust, as observed with anelectron microscope, is shown in Fig. 5. Analysis of
the composition of such particles showed that they
contained magnesium and silicon, suggesting that theparticles were the result of the chrysotile in the coating material melting and solidifying. It also suggeststhat irradiating asbestos with a laser can prevent itfrom being dispersed while it is still harmful. We
believe that the use of a diffractive optical element in
the manner described above can improve the efficiency of asbestos-removal operations.

Fig. 5. E lectron micrograph of a particle in dust generated
by laser irradiation.

irradiation conditions.

5. Summary
A compact, lightweight rust-removal tool—weighing only 500 g—using a high-power laser and diffractive optical element was developed for efficiently
removing rust from communication towers. We areresearching the application of this tool to the removalof asbestos, which involves melting the asbestos during the demolition and renovation of asbestos-containing buildings. We conducted an experiment inwhich asbestos was irradiated with a high-power
laser beam. The results of this experiment suggestthat asbestos can be melted and asbestos dust can besuppressed by appropriately controlling the laserNTT Technical Review

Vol. 20 No. 10 Oct. 2022

References
[1]

[2]
[3]
[4]
[5]

S. Kawamura, T. Sakamoto, M. Ueno, and S. Oka, “Derusting
Technology Using High-power Laser Device,” NTT Technical
Review, Vol. 19, No. 6, pp. 61–65, 2021.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr202106fa9.html
R. L. Virta, “Worldwide Asbestos Supply and Consumption Trends
from 1900 through 2003,” Circ. 1298, US Dep. Inter., USGS., Reston,
VA, USA, 2006.
J. LaDou, “The Asbestos Cancer Epidemic,” Environ. Health
Perspect., Vol. 112, No. 3, pp. 285–290, 2004.
L. Kazan-Allen, “Global Asbestos Panorama 2019,” 2019 Asbestos
Safety Conference, Perth, Australia, Nov. 2019.
Study Committee on Preventing Asbestos Dispersal, Central
Environment Council, Ministry of the Environment, “Current Statesand Issues for Preventing Asbestos Dispersal,” 2018 (in Japanese).
https://www.mhlw.go.jp/content/11201000/000422866.pdf

80Regular Articles

[6]

[7]

Bureau of Environment, Tokyo Metropolitan Government,“Inspection and Management Manual of Asbestos for Private
Buildings and Structures,” 2019 (in Japanese). https://www.kankyo.
metro.tokyo.lg.jp/air/air_pollution/emission_control/asbestos/manuals/kentikumanual.files/kentikuasubesuto_manual.pdf
M. Fujishige, R. Zinbo, K. Suda, Y. Sakuragi, I. Karasawa, and A.
Kojima, “Rapid Decomposition of Asbestos and with Various

Sohan Kawamura

Research Engineer, Smart Maintenance andManagement Technology Development Project,
Health and Environmental-Sensing Device Project, NTT Device Innovation Center.
He received a B.E., M.E., and Ph.D. in engineering from Hokkaido University in 2004,
2006, and 2009. He studied single crystal growthand scintillation properties of Ce-doped gadolinium pyrosilicate. He joined the Advanced
Opto-electronics Device Research Group in NTT
Photonics Laboratories in 2009, where he has
been studying crystal growth of electro-opticalmaterials, nonlinear optical devices, and diffractive optical elements for high-power lase machining.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

[8]

Additive,” The Gunma-Kohsen Review, No. 32. pp. 79–83, 2013.
S. Kawamura, S. Oka, K. Takaya, and M. Tsuda, “Detoxification andScattering Suppression of Asbestos-containing Material Using a
Laser Beam,” Proc. of SPIE 11994, High Power Laser Materials
Processing: Applications, Diagnostics, and Systems XI, 119940H,
Mar. 2022.

Kaori Takayanagi

Engineer, Sustainable System Group, Zero
Environmental Impact Research Project, NTT
Space Environment and Energy Laboratories.
She received a B.E. in precision engineeringand M.E. in environmental science from the
University of Tokyo in 2018 and 2020. She studied recycling systems for building waste materials. She joined the Sustainable System Group in
NTT Space Environment and Energy Laboratories in 2021, where she is engaged in research on
waste-disposal system for asbestos-containing
materials using laser.

81

Global Standardization Activities

Standardization Activities for OpticalFiber and Cable Technology inInternational ElectrotechnicalCommission
Yusuke Yamada, Taiji Sakamoto, Takashi Matsui,
and Noriyuki Araki
Abstract

The International Electrotechnical Commission Technical Committee 86 (IEC TC 86) is an international standardization organization that prepares and decides on international standards in relation to
products used for optical fiber telecommunication. In this article, we provide an overview of standardization activities, introduce topics discussed at the meetings in 2021 and 2022, and describe the Japanesestandardization strategy in IEC TC 86.
Keywords: IEC, optical fiber, international standard

1. Introduction
The International Electrotechnical Commission(IEC) is a standardization organization that sets all of
the required standards for electrical and electronics
technologies. In IEC, Technical Committees (TCs),
which are established for each technical field, holddetailed discussions on the content, publication, andrevision of international standards, and TC 86 is incharge of specifications and test methods of opticalproducts in optical communication systems. NTT is
researching and developing technologies and requirements for optical communication systems, and international standards are closely related to equipment
procurement and interconnection. Therefore, NTT is
actively participating in IEC TC 86 standardization
activities making proposals for international standardization, and involved in discussion of documents
to guarantee the quality and interoperability of telecommunication networks and product specifications
used in Japan. The International Telecommunication
Union - Telecommunication Standardization Sector
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

(ITU-T) Study Group (SG) 15 is also discussing
international standardization for optical fibers andcables from the viewpoint of requirements for public
communication network systems, and we are actively
participating in it. In IEC, standardization is mainly
conducted from the viewpoint of consistency between
procurement specifications for optical products andinternational standards, and in ITU-T, standardization is being discussed from the viewpoint of securing interoperability of communication systems andrequirements. Standardization activities in IEC and
ITU-T are being conducted in close cooperation witheach other.
In IEC, TC 86 is in charge of fiber optics (opticalfiber) technology, and its main purpose is to establish
standards such as optical, environmental, andmechanical requirements of fiber optics products for
optical communication systems, which are opticalmodules and devices used with optical fiber cables,
connectors, and communication equipment. The
technical fields of TC 86 are shown in Fig. 1, and theorganizational structure of TC 86 is shown in Fig. 2.
82

Global Standardization Activities

SC 86C
Fiber optic systems
and active devices

Submarine network

Core network
SC 86A
Fibers and cables

Metro network

Access network

SC 86B
Fiber optic interconnecting
devices and passive components

SC 86A

Fig. 1. Technical area of IEC TC 86.

Fibers and cables
WG 1 Product specifications & test procedures of optical fibers
WG 3 Product specifications & test procedures of optical cables

SC 86BWG 4 Standard tests and measurement methods
WG 6 Specifications for interconnecting devices and related components
WG 7 Specifications for fiber optic passive components
JWG 8 Product specifications of optical patch cords
Fiber optic systems and active devices
WG 1 Test methods and design of fiber optic communications systems

SC 86CTC 86 (Fiber optics)

Fiber optic interconnecting devices and passive components

WG 2 Sensing method using fiber optic products
WG 3 Product specifications & test methods of amplifiers and dynamic modules
WG 4 Product specifications & test procedures for fiber optic active devices
WG 4 Calibration procedure for optic test instruments
JWG 9 Optical functionality for electronic assemblies

Fig. 2. SCs and WGs of IEC TC 86.

TC 86 has three Subcommittee (SCs) to make decisions in each technical field. SC 86A is in charge of
optical fiber cables, SC 86B is in charge of opticalconnecting and passive devices, and SC 86C is incharge of optical subsystems and active devices. It
also has 12 Working Groups (WGs) to discuss specific standards documents. Each WG proposes new or
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

revised standards for the technical field it is in chargeof and prepares documents, which are then issued andreviewed by each SC. This article describes recent
trends in optical fiber and cable standards being discussed in IEC TC 86.

83

Global Standardization Activities

Coating

Bonded area

Optical fiber

Optical fiber

Conventional standard
250-µm coated fiber

200 µm(b) Partially bonded optical fiber ribbon

250 µm(a) Conventional optical fiber ribbon

Newly added standard
200-µm coated fiberStandard cladding diameter (125 µm)
Half thickness of conventional coating(c) Configuration of optical fiber ribbon with 200-µm coated optical fiber

Fig. 3. Configuration of optical fiber ribbon.

2. Recent trend in standardization for opticalfibers and cables
In the field of fiber optics for IEC TC 86, discussions on the standardization of optical cables andconnecting devices for access systems have been
active, especially against the background of the rapid
increase in demand for optical fiber cables in datacenters and the rapid spread of FTTH (fiber to thehome) in Europe and emerging countries. At SC 86A,
which handles optical fiber cable standards, revisions
are being made to optical fiber and optical fiber ribbon standards in line with the increasing number andhigh density of optical cables, and proposals anddiscussions are being made on various testing methods for optical cables in consideration of the usage
environment in each country.
The standardization of a partially bonded opticalfiber ribbon, which was developed and used for ultrahigh density optical cables in Japan, has been discussed. Figure 3 shows the configuration of an optical fiber ribbon. A partially bonded optical fiber ribbon can be discretely bonded in the longitudinal
direction to improve flexibility, such as being able toroll up or bend in any direction, and can be massspliced. By using the partially bonded optical fiber
ribbon, the optical fiber assembled density in the optical cable can be dramatically increased, and it is commercially used for optical fiber networks in Japan. In
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

IEC, the standard document was newly established asa product standard of optical fiber ribbon (IEC60794-1-31). It has recently been applied to ultrahigh fiber-count optical cables for inter/intra data
center networks. In ultra-high fiber-count opticalcables, it is effective in reducing the coating diameterof the optical fiber ribbon to make it smaller anddenser. In addition to the standard coating diameter of
250 μm, an alternative coating diameter of 200 μm
has been standardized. By applying this to opticalfiber ribbon, it is possible to reduce the diameter,
increase the density, and improve connection efficiency. In view of the situation in which connection
compatibility can be ensured by the progress of
fusion-splicing technology corresponding to 200-μm
coating, as shown in Fig. 3(c), the standard document
was published as a product standard of optical fiber
ribbon applying 200-μm coated optical fiber. For submarine optical fiber cables, it is necessary to increase
the number of optical fibers as communication
demand increases. We are also deliberating the application of 200-μm coating, which is an alternative
coating diameter, to cut-off shifted fiber (IEC 607932-50: B-654. A, B, and C) used for submarine applications.
The standards of test methods for optical fiber
cables are indispensable for proper characterizationof optical fiber cables. Figure 4 shows the numberingsystem of such test methods and the proposal for84

Global Standardization Activities

Conventional

Proposal for revision

IEC 60794-1-2: General guidance

IEC 60794-1-2: General guidance

Mechanical test methods
IEC 60794-1-21:
Mechanical test methods(Methods E1~34)

IEC 60794-1-1xx:

Environmental test methods
IEC 60794-1-22:
Environmental test methods(Methods F1~19)

IEC 60794-1-2xx:

Cable element test methods
IEC 60794-1-23:
Cable element test methods(Methods G1~11)

IEC 60794-1-3xx

ExamplesIEC 60794-1-101: Method E1 Tensile
IEC 60794-1-102: Method E2 Abrasion••

IEC 60794-1-201: Method F1 Temperature cycle
IEC 60794-1-202: Method F2 Water penetration••

IEC 60794-1-301: Method G1 Bend
IEC 60794-1-302: Method G2 Dimension••

Electrical test methods
IEC 60794-1-24:
Electrical test methods(Methods H1~3)

IEC 60794-1-4xx

IEC 60794-1-401: Method H1 Short circuit
IEC 60794-1-402: Method H2 Lightning••

Fig. 4. Numbering system of test methods for optical fiber cables.

revision. A wide variety of test-method standards are
divided into separate volumes to ensure timely maintenance of each standard. Japan proposed a test
method for evaluating the freezing resistance of optical fiber cables considering their installation in cold
regions on the basis of NTT’s knowledge, and published it as a freezing test method (IEC 60794-1-215:
Method E15C). Among the cable element test methods, Japanese experts are revising the document of
the test methods for ribbon dimensions and geometry(IEC 60794-1-302: Method G2, IEC 60794-1-303:
Method G3) and the test method for ribbon tear(separability) (IEC 60794-1-305: Method G5), which
are closely related to the product standards for opticalfiber ribbon, and deliberations are underway to makethem appropriate test-method standards.
In the research and development (R&D) process,NTT conducts evaluation using these test methods tofeedback to optical cable design and compatibility
with the usage environment, and examines performance and reliability evaluation in practical application development of new products and expansion of

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

application of developed products. Therefore, we arepromoting the globalization of technologies developed by NTT and Japan through standardization
activities in conjunction with R&D activities.
3. Further activities
Japan has aggressively promoted the domestic
deployment of optical-fiber-cable technology, andhas a strong presence as one of the major countries in
IEC TC 86. On the basis of our extensive experienceand knowledge in the introduction and operation of
optical fiber and cable technologies, we will contribute to the development of international standards that
contribute to the development of optical communication networks by closely cooperating with ITU-T,
which is also involved in international standardization. We will also promote the establishment of nextgeneration optical fiber and cable technologies,
which will be indispensable for IOWN (the Innovative Optical and Wireless Network), through both
R&D and standardization activities.

85

Global Standardization Activities

Yusuke Yamada

Takashi Matsui

Senior Research Engineer, Access Media Project, NTT Access Network Service Systems Laboratories.
He received a B.E. and M.E. in environmental
system engineering from Nagaoka University ofTechnology, Niigata, in 2003 and 2005. In 2005,
he joined NTT Access Network Service Systems
Laboratories, where he has been engaged inresearch on optical fiber cable. He has contributed to the activity of IEC SC 86B in 2013–2014
and IEC SC 86A since 2017. He is a member of
the Institute of Electronics, Information, andCommunication Engineers (IEICE) of Japan. He
received the best paper award at 12th OECC/
IOOC in 2007.

Distinguished Researcher, NTT Access Network Service Systems Laboratories.
He received a B.E., M.E., and Ph.D. in electronic engineering from Hokkaido University in
2001, 2003, and 2008. He attained the status ofProfessional Engineer (P.E.Jp) in electrical andelectronic engineering in 2009. In 2003, hejoined NTT Access Network Service Systems
Laboratories, where he has been researching
optical fiber design techniques. He has been
contributing to the activities of IEC TC 86 (fiber
optic systems) as an expert of SC 86A/WG1 andWG3 since 2009 and has been contributing tostandardization efforts in ITU-T SG15 since2022. He is a member of IEICE.

Taiji Sakamoto

Noriyuki Araki

Distinguished Researcher, NTT Access Network Service Systems Laboratories.
He received a B.E., M.E., and Ph.D. in electrical engineering from Osaka Prefecture University in 2004, 2006, and 2012. In 2006, he joined
NTT Access Network Service Systems Laboratories, where he has been engaged in research on
optical fiber nonlinear effects, low nonlinear
optical fiber, few-mode fiber, and multi-core
fiber for optical multiple-input-multiple-output
transmission systems. He is a member of Optica
and IEICE.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Director, Standardization Office, Research andDevelopment Planning Department, NTT.
He received a B.E. and M.E. in electrical andelectronic engineering from Sophia University,
Tokyo, in 1993 and 1995. He joined NTT AccessNetwork Service Systems Laboratories in 1995,
where he researched and developed operationand maintenance systems for optical-fiber-cable
networks. He has been contributing to standardization efforts in ITU-T SG6 since 2006. He was
the rapporteur of Question 6 of ITU-T SG6 from2006 to 2008 and rapporteur of Question 17 of
ITU-T SG15 from 2008 to 2012. He also served
as the chairman of the ITU-T Focus Group on
Disaster Relief Systems and Network Resilienceand Recovery. He was the vice-chairman of
ITU-T SG15 from 2013 to 2022. He also contributes to the activities of IEC TC 86 (fiber optic
systems). He received the ITU-AJ award fromthe ITU Association of Japan in 2017. He is amember of IEICE.

86

Practical Field Information about
Telecommunication Technologies
Practical Field Information about Telecommunication Technologies

Case Study of Audible Noise on
Analog Leased Lines Caused byElectromagnetic DisturbanceTechnical Assistance and Support Center, NTT EAST
Abstract

If electromagnetic disturbances, which are generated from malfunctioning appliances or radio waves,interfere with metallic cables, it can cause telecommunication failures such as audible noise being heard
when using telephones and disconnection of digital subscriber lines. This article provides a case studyof audible noise interfering with voice-based services, the cause of the electromagnetic disturbances, anda countermeasure against them. This is the seventy-second article in a series on telecommunication technologies.
Keywords: electromagnetic disturbance, PBX, Noise Search Tester

1. Introduction
If electromagnetic disturbances generated from
malfunctioning appliances or radio waves interferewith metallic cables, they can cause telecommunication failures such as audible noise being heard whenusing telephones, disconnection of digital subscriberlines, and transmission errors on ISDN (integrated
services digital network) lines. To resolve such failures, it is necessary to identify the source of the electromagnetic disturbances and eliminate them. This
article provides a case study of audible noise interfering with voice-based services, the cause of the electromagnetic disturbances, and a countermeasureagainst them.
2. Case study
A customer using a 3.4k analog leased line reported
that a high-pitched audible noise could be heard whenusing certain telephones. Though the local telecommunication maintenance personnel applied countermeasures, including changing the line number; theNTT Technical Review

Vol. 20 No. 10 Oct. 2022

audible noise was not cleared. Therefore, the Technical Assistance and Support Center (TASC), NTT
EAST was asked to investigate the problem.
2.1 Configuration of equipment and failure state
The customer operates their own private branchexchange (PBX) and uses 3.4k analog leased lines
between the PBX in the customer’s main building andseveral offices located in the same city or surrounding
areas, for extension calls. The configuration of theequipment is shown in Fig. 1. A redundant configuration is adopted between the customer’s main building(in which the customer’s PBX is installed) and an
NTT building (telecommunication central office),
having two different telecommunication accessroutes in the overhead section, so that the 3.4k analog
leased lines can be maintained even in the event of acable failure in the access network section. The routecan be switched by changing the jumper cable in themain distribution frame (MDF) in the customer’s
main building. There are two routes between the NTT
building connecting to the customer’s main building
and customer’s offices: (1) a route turning at the MDF
87

Practical Field Information about Telecommunication Technologies

Audible noise present on lines
housed in the same cable
Customer’s main building

100-paircable

MDF
Devicehousingleased line

Customer’s
own PBX

MDF
Audiblenoise

NTT building
200-pair cable
Approx.
2.1 kmApprox.
1.6 km

Two routes inoverhead section
Not normallyused

Configuration (1)Turning at the NTT building

Audiblenoise

ArresterOffice A

Devicehousingleased line

Another
NTT building

Audiblenoise

ArresterOffice B
Configuration (2)Going through anotherNTT building

Fig. 1. Configuration of equipment.

in the NTT building and (2) one going through another NTT building in the city.
The customer reported that high-pitched audible
noise was being constantly heard through the telephones installed at multiple offices. The results ofon-site interviews with the customer revealed that the
audible noise was present on four lines.
To confirm the details of the noise, on-site telecommunication maintenance personnel investigated all
300 lines installed in the customer’s main building
using a cable tester, and it was confirmed that the
audible noise was present on 17 lines of the 200-paircable in use. No audible noise was generated on linesof the other 100-pair cable. As a countermeasure, thetelecommunication maintenance personnel changed
a line number within the same 200-pair cable, but the
audible noise was not cleared, so the cable wasswitched to the stand-by 100-pair cable in the redunNTT Technical Review

Vol. 20 No. 10 Oct. 2022

dant configuration, and the audible noise was cleared.
Because the cause of the audible noise was unknown,
and the audible noise was generated again when theroute was switched back to the 200-pair cable due toconnection failure, etc., TASC was asked to investigate the cause of the audible noise.
2.2 Investigation of the cause of audible noise
TASC first investigated the cause of the audible
noise. TASC asked the on-site telecommunication
maintenance personnel to check the condition of theequipment and measure the audible noise at the site.
On the basis of the results obtained on-site, TASC
analyzed the measurement results, determined thecause of the noise from the analysis results, and proposed a countermeasure.
(1)	Confirmation of frequency characteristics andcycle of the audible noise88Practical Field Information about Telecommunication Technologies

120

FFT analysis
Around 800 Hz Around 2.4 kHz
Sampling frequency: 44,100 Hz
Number of samples: 59,616,000

Level (dB)

100
80
60
40
20
0
−20
0

10

100

1k

10 k

Frequency (Hz)Fig. 2. Results of FFT analysis.

To confirm the frequency characteristics of the
audible noise reported by the customer, it was recorded at the site. The recorded sound files were subjected
to First Fourier Transform (FFT) analysis by TASC.
The results of this analysis are shown in Fig. 2. It was
found that the level of the noise signal was higher at
the fundamental frequency (which is approximately
800 Hz) and its harmonics at odd multiples. It wasalso found that the audible noise was occurring periodically (every 20 to 30 s).
(2) Identification of noise source
To identify the lines with the highest audible noise,
the voltage levels of the audible noise on 17 lines ofthe 200-pair cable routed into the customer’s mainbuilding were measured. The Noise Search Tester(3144) and its probe, i.e., a voltage sensor (9741),
both manufactured by Hioki E.E. Corporation(Photo 1), were applied to measure the commonmode voltage levels of the audible noise at the MDF
of the NTT building. The basic specifications of theNoise Search Tester are listed in Table 1. Commonmode voltage levels for seven frequency ranges canbe displayed by the tester’s bar indicator.
The measurement results of the lines with the highest common-mode voltage level are shown in Fig. 3.
It was found that the highest common-mode voltage
level (about –2 dBV) occurred in the 1 kHz band, i.e.,
voltage level at frequencies between 500 Hz and 1.5
kHz.
Since this frequency range includes 800 Hz, i.e., thefundamental frequency of the recorded sound file,
TASC concluded that it corresponds to the commonmode voltage level of the audible noise reported by
the customer. It was also found that the commonmode voltage level fluctuates periodically and correNTT Technical Review

Vol. 20 No. 10 Oct. 2022

Photo 1. N oise Search Tester 3144 (right) and voltage
sensor 9741 (left).

sponds to the phenomenon observed by playing backthe sound file, i.e., a high-pitched noise (at approximately 800 Hz) occurring periodically every 20 to
30 s.
2.3 Estimation of source of audible noise
The repetition of the sound at approximately 800
Hz suggests the “howler (off-the-hook) tone” of the
PBX. A howler tone is an audible sound transmitted
when a telephone is off the hook for an extended
period, even though the line is not used. We examinedthe conditions under which the customer’s PBXtransmits a howler tone (Table 2) and found that thefrequency, transmission time, and repetitive transmission conditions of the howler tones were almost identical to those of the audible noise identified in theinvestigation.
Since the PBX in question transmits a howler tone89

Practical Field Information about Telecommunication Technologies

Table 1. Basic specifications of Noise Search Tester.
• Dedicated terminal 9741
• BNC terminal

Input terminal
Maximum input voltage

5 V peakMeasurement range• ×1 range: 0 to –30 dBV• ×10 range: –20 to –50 dBVFrequency bandwidth• 100 Hz to 30 MHz, separated into seven bands

Measurement-range setting andmeasurement-frequency bandwidth (–3 dB)(Measuring range: frequency bandwidth)• 1 kHz: 500 Hz to 3 kHz• 15 kHz: 7.5 kHz to 22.5 kHz• 70 kHz: 5 kHz to 105 kHz• 250 kHz: 125 kHz to 375 kHz• 1 MHz: 0.5 MHz to 1.5 MHz• 3 MHz: 1.5 MHz to 4.5 MHz• 20 MHz: 10 MHz to 30 MHz

Common-mode voltage level (dBV)

BNC: Bayonet Neill-Concelman

Approx. 25 s

Approx. 30 s

Frequency bandwidth
1 kHz
15 kHz70 kHz1

2

3

250 kHz
1 MHz
3 MHz
20 MHz4(Min.)

Fig. 3. Measurement results of common-mode voltage levels of audible noise using Noise Search Tester.

Table 2. Conditions for transmitting howler tone.
Howler tone

Frequency: 800 Hz
Level: Max. +23 dBm (approx. 11 V)

Conditions

Detection of loop current of 13 mA or more

Howler-tone settings

Transmission for 30 s
(repeated howler tones can be transmitted)

when it detects a loop current of 13 mA or more, thecause of the audible noise is assumed to be faulty
insulation on the line connected to the PBX. The
cause of the audible noise on multiple lines isassumed due to the deterioration of the balance of theline due to faulty insulation of the line, which induced
the howler tone being sent to the line in question to be
directed to an adjacent line.
NTT Technical Review

Vol. 20 No. 10 Oct. 2022

2.4 V
 erification of countermeasure and its effectivenessThe line with the highest audible noise was a standby line that had been taken out of service due to thereconstruction of office B, but the jumper cable in the
NTT building had been left connected. When thestand-by line connected to the MDF in the NTT
building was disconnected, the audible noise on90

Practical Field Information about Telecommunication Technologies

multiple lines was eliminated.
3. Concluding remarks
A case study of audible noise on a 3.4k analog line
and the countermeasure against it were introduced.
The local maintenance personnel confirmed thedetails of the equipment configuration, recorded the
audible noise in the customer’s environment, andmeasured the voltage levels of the audible noise to
identify its mechanism. The measurement results
were analyzed at TASC, and a countermeasure based

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

on the results was applied to clear the audible noise.
TASC would like to thank the local maintenance personnel for their cooperation.
The EMC Engineering Group in TASC continues toactively engage in technical cooperation, development, and dissemination of its technology throughactivities such as technical seminars to reduce telecommunication failures caused by electromagnetic
disturbances, radio waves, induction, lightning, andother factors and to improve the reliability of telecommunication services.

91External Awards
Honorable Mention Award
Winners: Hiroki Koyama, Chiba Institute of Technology; Yuuna
Nakagawa, Chiba Institute of Technology; Shigeaki Tanimoto, Chiba
Institute of Technology; Teruo Endo, Osaka Shoin Women’s University; Takashi Hatashima, NTT Social Informatics Laboratories; andAtsushi Kanai, Hosei University
Date: July 2, 2022
Organization: International Institute of Applied Informatics (IIAI)
For “A Study of Risk Assessment Quantification for Secure Telework.”
Published as: H. Koyama, Y. Nakagawa, S. Tanimoto, T. Endo, T.
Hatashima, and A. Kanai, “A Study of Risk Assessment Quantification for Secure Telework,” Proc. of the 12th International Congress
on Advanced Applied Informatics (IIAI-AAI 2022), pp. 574–580,
Kanazawa, Japan, July 2022.
Best Paper Award
Winners: Wataru Kobayashi, NTT Device Technology Laboratories;
Shigeru Kanazawa, NTT Device Innovation Center; Takahiko Shindo, NTT Device Innovation Center; Manabu Mitsuhara, NTT Device
Technology Laboratories; Fumito Nakajima, NTT Device Technology Laboratories
Date: July 6, 2022
Organization: The 27th OptoElectronics and Communications Congerence (OECC 2022)
For “1.5 pJ/bit, 128 Gb/s, 50°C Operation of AXEL for Short
Reach Application.”
Published as: W. Kobayashi, S. Kanazawa, T. Shindo, M. Mitsuhara,
and F. Nakajima, “1.5 pJ/bit, 128 Gb/s, 50°C Operation of AXEL for
Short Reach Application,” OECC 2022, WD1-2, Toyama, Japan, July
2022.
Best Paper Award
Winners: Tomoyuki Kato, Fujitsu Ltd.; Hidenobu Muranaka, Fujitsu
Ltd.; Yu Tanaka, Fujitsu Ltd.; Yuichi Akiyama, Fujitsu Ltd.; Takeshi
Hoshida, Fujitsu Ltd.; Shimpei Shimizu, NTT Network Innovation
Laboratories; Takayuki Kobayashi, NTT Network Innovation Laboratories; Takushi Kazama, NTT Network Innovation Laboratories/
NTT Device Technology Laboratories; Takeshi Umeki, NTT Network Innovation Laboratories/NTT Device Technology Laboratories; Kei Watanabe, NTT Network Innovation Laboratories/NTT
Device Technology Laboratories; Yutaka Miyamoto, NTT Network
Innovation Laboratories
Date: July 13, 2022
Organization: OECC 2022
For “WDM Transmission in S-band Using PPLN-based Wavelength Converters and 400-Gb/s C-band Real-time Transceivers.”
Published as: T. Kato, H. Muranaka, Y. Tanaka, Y. Akiyama, T.
Hoshida, S. Shimizu, T. Kobayashi, T. Kazama, T. Umeki, K. Watanabe, and Y. Miyamoto, “WDM Transmission in S-band Using
PPLN-based Wavelength Converters and 400-Gb/s C-band Real-time
Transceivers,” OECC 2022, WA3-4, Toyama, Japan, July 2022.

NTT Technical Review

Vol. 20 No. 10 Oct. 2022

Young Researcher Award
Winner: Tatsuya Iizuka, NTT Space Environment and Energy Laboratories
Date: July 15, 2022
Organization: Information Processing Society of Japan (IPSJ)
For “Corner-reflector-based Chipless RFID for Wide Range Readout with Low Power Millimeter Wave Radar.”Published as: T. Iizuka, T. Sasatani, N. Kosaka, M. Hisada, K.
Narumi, and Y. Kawahara, “Corner-reflector-based Chipless RFID
for Wide Range Readout with Low Power Millimeter Wave Radar,”Proc. of Multimedia, Distributed, Cooperative, and Mobile Symposium (DICOMO) 2022, 5H-3, pp. 1085–1091, July 2022.
IPSJ Yamashita SIG Research Award
Winner: Toshiki Shibahara, NTT Social Informatics Laboratories
Date: July 29, 2022
Organization: IPSJ
For “Privacy Risk of Differentially Private Bayesian Neural Network.”
Published as: T. Shibahara, T. Miura, M. Kii, and A. Ichikawa, “Privacy Risk of Differentially Private Bayesian Neural Network,” Proc.
of Computer Security Symposium 2021, pp. 245–252, Oct. 2021.
Best Paper Award
Winners: Xuejun Xu, NTT Basic Research Laboratories; Masaya
Hiraishi, University of Otago; Tomohiro Inaba, NTT Basic ResearchLaboratories; Tai Tsuchizawa, National Institute of Advanced Industrial Science and Technology; Atsushi Ishizawa, Nihon University;
Haruki Sanada, NTT Basic Research Laboratories; Takehiko Tawara,
Nihon University; Jevon Longdell, University of Otago; Katsuya
Oguri, NTT Basic Research Laboratories; and Hideki Gotoh, NTT
Basic Research Laboratories/Hiroshima University
Date: August 3, 2022
Organization: The 15th Conference on Laser and Electro-Optics
Pacific Rim (CLEO-PR 2022)
For “Erbium-doped Rare-Earth Oxide Thin Film Waveguides for
Integrated Quantum Photonic Devices.”
Published as: X. Xu, M. Hiraishi, T. Inaba, T. Tsuchizawa, A.
Ishizawa, H. Sanada, T. Tawara, J. Longdell, K. Oguri, and H. Gotoh,“Erbium-doped Rare-Earth Oxide Thin Film Waveguides for Integrated Quantum Photonic Devices,” CLEO-PR 2022, July/August
2022.
Oral Talks Award
Winner: Motoki Asano, NTT Basic Research Laboratories
Date: August 11, 2022
Organization: The 18th International Workshop on NanomechanicalSensing (NMC2022)
For “An Optomechanical Prober in Liquid with Twin Microbottle
Resonators.”
Published as: M. Asano, H. Yamaguchi, and H. Okamoto, “An Optomechanical Prober in Liquid with Twin Microbottle Resonators,”
NMC2022, Aug. 2022.

92