NTT Technical Review

February 2019 Vol. 17 No. 2
View from the Top
 Motoyuki Ii, Senior Executive Vice President and Head of Technology Planning, NTT
Feature Articles: Keynote Speeches at NTT R&D Forum 2018 Autumn
 Jun Sawada, President and Chief Executive Officer, NTT Katsuhiko Kawazoe, Senior Vice President, Head of Research and Development Planning,NTT
Feature Articles: Value Creation by Leveraging State-of-the-art Research Results
 Ensuring Greater Safety for Our Firefighters and Our Communities: Integrating FLAIM
Trainer™ and hitoe™
 Utilization of NTT’s R&D Technology in the Financial Industry
 Initiatives in the Primary Industries (Agriculture, Forestry, and Fisheries) Utilizing Advanced
Technology of the NTT Group Improving Amenities at Narita International Airport—Utilizing High-precision Indoor Mapsand Geomagnetic Positioning to Facilitate the Movement of Visitors
 Creating Value from Deep Learning Technology and Its Business Applications
Regular Articles
 Highly Efficient Gate Controllability of Rashba Spin-orbit Interaction in a Gate-all-around
InAs Nanowire MOSFET
Global Standardization Activities New Standardization Trends at GlobalPlatform—Secure Components for the IoT Era
Practical Field Information about Telecommunication Technologies
 Investigation of the Cause of Disconnection of Call Parking on a Business Phone
Information
 Report on NTT R&D Forum 2018 Autumn
Short Reports
 Controlling Nuclear Magnetic Resonance with a Micromechanical Oscillator—A Novel
Technique to Individually Manipulate Nuclear Spins in Integrated Devices ExternalAwards/Papers Published in Technical Journals and Conference Proceedings
External Awards/Papers Published in Technical Journals and Conference Proceedings

View from the Top

Boldly Taking the Helm to Be “Your
Value Partner” Now and into the
FutureMotoyuki Ii
Senior Executive Vice President and Head ofTechnology Planning, NTT
Overview

Innovation in digital technologies is disrupting existing industries while
creating new value. At the same time, social phenomena such as an aging
population with a declining birthrate and a shrinking labor force continue
to have a major impact on consumer lifestyles and corporate activities inJapan. NTT has formulated a digital transformation strategy in the face ofsuch social issues. We asked NTT Senior Executive Vice President Motoyuki Ii, a driving force behind this strategy, to tell us how he plans to deal withthese issues as Chief Digital Officer.
Keywords: digital transformation, disruption, R&D

Put employees at ease, accumulate small
successes, and pursue bold changes—NTT Group’s Medium-Term Management Strategy
has recently been announced. How do you plan toimplement this strategy as Senior Executive Vice
President and Chief Digital Officer?
The theme of this Medium-Term Management
Strategy is bringing about a major transformation in
NTT and the NTT Group. There are four pillars making up this strategy, two of which promote our customers’ digital transformations and our own digitaltransformation. This is my mission as Chief Digital
Officer (CDO). The position of CDO was newly
established in August 2018 not only in the holding
company but also in major NTT Group companies
i n c l u d i n g N T T E A S T, N T T W E S T, N T T
Communications, NTT DOCOMO, and NTT DATA.
Each company’s CDO has the job of maximizing the
NTT Technical Review

Vol. 17 No. 2 Feb. 2019use of data and information and communication technology (ICT) to revolutionize business processes andmake human intervention minimal. In other words,
each CDO has the role of bringing about a majortransformation in management. Here, transformationmeans changing existing processes, or to put it another way, to disrupt the current way of doing things.
There will naturally be some resistance from people
who have been working in the same way for years, so
to bring about a transformation, it is essential that a
CDO “take the helm” with good negotiating skills
and courage without yielding to that resistance.
Although the original meaning of “D” in “CDO” isdigital, it can also stand for disruptive, reflecting therole of the CDO as the person with ultimate responsibility for disruptive innovation.

1

View from the Top—A CDO takes on the heavy responsibility of overseeing a digital transformation, but to push suchchange through, won’t a good understanding of current conditions be needed?
A CDO can also be appointed from the outside.
That person would have to have a detailed knowledge
of ICT and be proficient in business analysis, andshould also be quite familiar with the way things havebeen done up to now. A company is a living thing, so
it is essential that a CDO understand the present way
of doing things and be able to establish a strategy for
determining how things must change going forward.
For this reason, the NTT Group is appointing CDOs
from among people who have held responsible positions requiring an extensive understanding of business and work and who have much and varied experience in their backgrounds. On top of all this, a CDO
must pursue major changes while accumulating small
successes over big ones to put employees at ease. A“quantum leap” in the way things are done can be
stressful for anyone, so employees are more likely to
oppose such change than to give their approval. It isvitally important here to draw up a goal that everyone
can commit to and to then move toward that goal one
step at a time. To repeat, people are bound to show
resistance to change, so a CDO must understand how
to get employees to move forward to achieve transformation.
That being said, I have been involved in all sorts ofwork since entering NTT about 35 years ago, including technology development, international procurement, plant planning, personnel affairs, NTT reorganization, and corporate sales, and I have experienced
many successes and failures, all of which should
serve me well as a CDO.
The mission of a CDO is to promote the use of newtechnologies and systems to automate various typesof business processes from a management perspective. It is precisely for this reason that a CDO who
cannot grasp where manpower is realistically andspecifically being applied will not produce effective
results. As a CDO, I believe I can put my experiencesin problem solving in many different areas to gooduse.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

There is no such thing as unfinished work. The
only recourse is to face a problem with resolve,
honesty, and sincerity.
—You said you have faced many situations involving
problem solving, but can you tell us what area of yourwork taught you the greatest lesson?
Let me tell you about a matter that I think I can callone of the biggest crises in my life. It occurred during
the four years that I served as an NTT EAST senior
executive vice president and head of corporate sales.
It involved the use of digital technology to make amajor change to the system of a certain university. We
were using new technology that we were not yet fully
adept at, and as a result, we missed the delivery date
and were unable to achieve some functions we promised, all of which caused our customer a great deal oftrouble. Repairing the system would require a period
measured in years, and the whole project ended up in
the red.
Although this work involved collaboration withNTT Group system integrators and participatingcompanies such as vendors, NTT EAST was theprime contractor, making us responsible for the project; we could not blame anyone else. In such a situation, the only approach was to be honest and sincere
with the customer. Making excuses did not erase thefact that the problem was caused by a lack of technical competence and that the deadline could not be
met. We gave the customer an honest assessment as towhat efforts or measures we were taking, what problems still remained, and about how much time would
be needed to rectify those problems. I took this action
thinking that it was very important that I be viewed as
an honest person and as someone who could be2

View from the Top

trusted. In addition, as leader of this project, I madean effort to cheer up and embolden members of mystaff who were working for days on a major problem
and being yelled at by the customer. This was truly along battle that could not have been won without thestrong bonds of a team that would not give up. Lost
time cannot be recovered, so the only thing to do ismove forward. Convinced that we could solve theproblem and be accepted by the customer once andfor all, all the members of this team confronted thissituation with great resolve. It was a good opportunity to gain hands-on experience about learning fromfailure.
—I can sense a terrible experience by just listening to
your story. All of your challenging experiences
should prove to be useful in your upcoming work.
I believe that experience is the driving force behind
one’s personal growth. It is important to have many
experiences and make maximum use of them.
However, the number of experiences one can have islimited, so reading can make up for that shortfall as
pseudo experiences. Incorporating the experiencesthat others have had can deepen one’s knowledge, so
I encourage my employees to read as many books as
possible.
I use as a reference the many conflicts that arose
during Japan’s Muromachi period (1338–1573) andbeyond when feudal warlords vied for politicalpower, and I am also inspired by historical novels that
depict peaceful times during the Edo period (1603–
1867) when culture flourished. In particular, I find it
deeply interesting how Hojo Soun established thefoundation for the Hojo clan. Similarly, the Edo

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

period originates with Tokugawa Ieyasu, Toyota
Motor Corporation with Sakichi Toyoda, andMatsushita Electric Industrial Co., Ltd. (nowPanasonic Corporation) with Konosuke Matsushita.
Stories about people known as a chukonoso, whohelped to make a corporation thrive, serve as a reference in my work.
I also find books that make predictions about the
future quite interesting. For example, I recommend“Homo Deus: A Brief History of Tomorrow” by thebestselling author Yuval Noah Harari, also known for“Sapiens: A Brief History of Humankind.” I also
liked “The Singularity Is Near: When Humans
Transcend Biology” by the American scientist Ray
Kurzweil. Now 70 years old, Ray Kurzweil wasplaced in charge of developing artificial intelligenceat Google six years ago. From these books, I learned
that as science and technology develops, humans
themselves will undergo dramatic changes as they
assimilate those advances. This refers not only to thegrowth of humans as a species from the perspectiveof natural science but also to changes in individual
humans due to science and technology that we ourselves created. Given the view that 2045 will mark atransition from Homo sapiens to post-humanism, it
can be seen that we are living in a period in whichmankind itself is undergoing a transformation. On
taking charge of transforming the company in such aperiod and thinking about the best approach to changing the way we work and the role of the company, I
feel that it is also necessary to consider the transformation in human beings.
In this way, I think it’s important that we compensate for any lack of experience by putting the knowledge of trailblazers and researchers to good use in ourwork. One man alone cannot achieve a successfultransformation, so I am taking up the challenge
together with the CDOs of the other NTT Group companies. To begin with, we will undertake a transformation achieved through software and robotics byanalyzing work processes and introducing ICT.
However, making immediate and straightforward
improvements to work processes will not be the endof our efforts. Our ultimate objective is to create an
ecosystem in which work is completed without
human intervention.
I am beginning this endeavor by sharing a roadmap
and goals among all CDOs. Perhaps there are somepeople who imagine that this is going to be a very
difficult endeavor based solely on the documents
describing these goals. Nevertheless, I would like todescribe this dream in the form of specific goals and3View from the Top

to consider how best to achieve them. I envision thesegoals to be achieved around the year 2050. In general,such plans for the future should be made, say, 100
years in advance, but as can be seen from the development of smartphones, technical innovation can bringabout changes that occur too fast to predict. Taking
such speed of technical development into account, I
think we can set goals about 30 years in advance.
Supporting people who take on difficult themes
and creating an appealing workplacefor employees—What is “work” from your point of view?
As I mentioned in my last interview (October 2016
issue), there is a difference between work and tasks.
Work is making discoveries on one’s own and opening up new possibilities. Carrying out something thatis simply given to you is not work. The action of
thinking about what problems exist and what goals toset, and aiming for those goals are what constitutes
work. Anything else we can regard as tasks. To put it
another way, work is taking up the challenge of amore difficult theme and making an effort that canlead to results.
I like the attitude of doing whatever it takes without
giving up and making an all-out effort with not only
the results in mind. If such an attitude can be maintained, some sort of achievement will be forthcoming. A theme can sometimes appear by chance, andsome themes can be difficult to achieve, while otherthemes can be easy to achieve. It’s simple to evaluate
things only based on the results, but I want to support
those who do not avoid difficult themes but rather
confront them head on. I am always telling thoseunder me that such passion is what makes a company
grow.
I think that a person in top management cannot succeed without having skills in expressing one’s vision
and without having decisiveness and courage.
Although one method is to allow for proposals from
subordinates in a bottom-up approach, I feel that presenting a clear direction from the top is one way ofmaking it easier for subordinates to do their work.
Failure to clearly indicate direction from the top
forces subordinates to guess what that direction is andleads to a company on shaky ground. It is essentialthat top management take a bold stance in presenting
direction. To execute that direction, it is necessary for
subordinates to have the attitude in which they
believe “I must think of another way to do this instead
NTT Technical Review

Vol. 17 No. 2 Feb. 2019of giving a reason for not being able to do it.” Thereis no top executive who will listen to a reason for not
being able to do something. For instance, with mountain climbing as an example, there are usually many
paths to the top, so a good one can be searched for,and if by chance there are no paths, one can be created. A quick ascent may be possible if one is skillful,or a struggle may ensue. Either way, it is importantthat one does not give up on reaching the top.
—Mr. Ii, what would you like to say to our researchers?
To begin with, I’d like to point out that I am concurrently serving as Chief Technology Officer with the
responsibility of establishing the NTT Group technology strategy, research and development (R&D)
direction, and a strategy for determining how to apply
our intellectual property to what kinds of services. In
recent years, technology in the information technology field has been advancing exponentially. However,people tend to think of things in a linear manner,
resulting in a gap with technology development. In
this regard, a mainstay of NTT R&D has been thecreation of intellectual property on its own. However,creating intellectual property in an independent manner may broaden that gap instead of closing it up.
Therefore, I would like to ask researchers to be proactive in adopting a co-innovation approach, whichmeans finding and incorporating intellectual propertyof value, integrating it with NTT’s own intellectual
property, and creating new value. As I just mentioned,
technology is advancing at an exponential rate, so I
would like to promote co-innovation and new value
creation so that we can ride the wave of technology
development.
Globalization is progressing not only in business4View from the Top

but also in the field of R&D, resulting in a broadeningof researchers’ activities. Amid this development,international headhunters have been targeting ourresearchers. In one sense, this is gratifying since it
offers proof that NTT has superior, top-notch
researchers. We are therefore revising our personnel
system to persuade such top researchers to stay at
NTT despite such invitations.
At the same time, NTT is opening research laboratories outside Japan and establishing environments inwhich our researchers can work together with top
international researchers. This represents a majorchange in direction for NTT R&D. In addition,
opportunities for joint research are expanding withsites that promote cutting-edge research or so-called“crazy research” not only in Japan but also in Europe,
the United States, and Israel. Such research activities
can increase the appeal of NTT, and conversely
improve the chances that NTT researchers are selected as partners in joint research. Achieving impressive
research results is important, but what we also reallywant to do is to create an attractive company for
researchers.
—Finally, could you leave a message for all NTT
employees?
Yes, I would like to ask everyone to have a variety
of experiences. Be curious about all sorts of things inrelation to work and play. And by all means, readbooks to make up for what your experiences do not

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

cover. The knowledge contained in books can inspireand enlighten you. When Internet searches leave you
unsatisfied, you can find what you need in books. A
book compiles the personal experiences and thoughtsof the author, and as such, it can stimulate and motivate the reader. I believe that obtaining information
and reading books are completely different worlds.
Of course, it’s difficult to make time when you arebusy, but life passes by quickly. I wouldn’t say one
book a day, but if you can read one book a week, thatwould be about 50 books worth of inspiration a year.
Please give it a try!

Interviewee profile
Career highlights
Motoyuki Ii joined Nippon Telegraph andTelephone Public Corporation (now NTT) in
1983. After becoming Manager of NTT EAST
Niigata branch office in July 2007, he became
NTT EAST Senior Vice President and concurrent
Executive Manager of the Plant Department andStrategy Planning Department, Network Business
Headquarters, in June 2011. He then became
NTT EAST Senior Executive Vice President andSenior Executive Manager of Corporate Sales
Promotion Headquarters in June 2016. He took
up his present position in June 2018.

5

Feature Articles: Keynote Speeches at NTT R&D
Forum 2018 Autumn

Bringing the Smart World to Life
Jun Sawada
President and Chief Executive Officer,
NTT
Overview

This article introduces an NTT Group effort in new value creation tobring the smart world to life. The contents of this article are based on alecture presented by Jun Sawada, NTT President and Chief Executive
Officer, at NTT R&D Forum 2018 Autumn held November 29–30, 2018.
Keywords: smart world, B2B2X, cognitive foundation, Your Value Partner 2025

1. Smart city
An NTT Group objective is to help create a prosperous and happy future for everyone. With the aimof achieving smart cities, we have been working forsome years on creating new value by utilizing data
together with the cities of Sapporo, Fukuoka, andYokohama through collaborations with industry, government, and academia. In this article, I will introduce NTT Group initiatives for the City of Las Vegas,
where smart city initiatives are taking root and evolving, and new initiatives toward urban development
and problem solving in Japan.
1.1 City of Las Vegas
The NTT Group has been conducting joint trials on
smart city solutions in the City of Las Vegas in partnership with Dell Technologies since September
2018 [1]. Our aim here is to construct a system thatcan detect the occurrence of incidents and help prevent crime by networking many cameras and sensors
installed in downtown Las Vegas and analyzing
pedestrian and traffic patterns.
This system is based on a model that combines elements that perform recognition and detection at the
network edge and elements that analyze and predict
trends by artificial intelligence (AI). It features a variety of advanced technologies from the NTT laboratoNTT Technical Review

Vol. 17 No. 2 Feb. 2019ries including sensing technologies. A new software
platform called Cognitive Foundation® has been prepared to interconnect these hardware and software
elements (Fig. 1). The plan going forward is toexpand the lineup of system functions to include
solutions, traffic monitoring, and other functions thatcan provide an even higher level of public safety. We
are also working on smart city initiatives in the State
of Nevada [2]. Going forward, we plan to roll out
smart city solutions throughout the United States incollaboration with Dell Technologies and NTT
Group sales teams in the United States.
The NTT Group has been promoting the businessto-business-to-X (B2B2X) model as shown in Fig. 2.
In this model, the NTT Group and its partners, whichcorrespond to the first ‘B’ in B2B2X, seek to create
new value by supporting main players, the second ‘B’in B2B2X.
The City of Las Vegas corresponds to the second‘B,’ and NTT and Dell Technologies, the first ‘B,’ do
not own the collected data. The City of Las Vegas
owns and utilizes such data, and our role is to support
the creation of new value. Michael Sherwood, Director of Information Technologies for the City of Las
Vegas, explained that the reason for selecting NTT
and Dell Technologies was not only our specialized
know-how in new data-use fields such as cybersecurity and data management, which Las Vegas needs for6

Feature Articles

Rapid and accurate situational awareness on site
Public safety solutions

Future development

Urban areas/event venues

Video cameras/
microphones

Area expansion• Other cities in Nevada• Roll out to other states, globally

Cognitive Foundation®
ICT resource orchestration
Reactive
Proactive

Function enhancement• Smart traffic• Monitoring of public parks• Audience attribute analysis for
digital signage and more

Detection andrecognition via edge
Trend analysis andforecasting

Incident/accident
response
Forecast response

MOU signed
State of
NevadaCity ofLas Vegas

NTT

MOU: Memorandum of Understanding

Fig. 1. Smart city initiative with City of Las Vegas.

B

NTT
Group

B

X

Serviceproviders(Main players)

New value
creation

Partners

Fig. 2. Direction of initiatives for B2B2X model.

development, but also our proposal that the City of
Las Vegas be a main player (an owner of the collecteddata). Data will be a basic resource for business andservices, indeed for everyday life in society. In Japan,
I think it should be local governments that own andcontrol such data. I would like to roll out our smart
city initiatives with this approach to data ownershipon a global scale.
1.2 Urban solutions company
NTT completed its tender offer bid for NTT Urban
Development Corporation, enabling NTT to acquire
95.2% of the company’s shares. Our aim here is toassist in urban development by combining the urban
development expertise of NTT Urban Development
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

and the construction- and power-related solutions of
NTT Facilities with NTT Group assets and othertechnologies. We hope to achieve diversity, innovation, resilience, and communities by using information technology (IT) and communications as essential
tools. I believe that by assisting in urban solutions inthis way and broadening our scope as a business
entity, we can contribute to society and the world atlarge.
2. Smart world
I introduce some examples concerning the smart
world here.

7

Feature Articles

Integration of traditional performing art and ICT2016 - 2018
Cho-Kabuki supported by NTT

2016 - 2018
Joint experiment to propose new ways to enjoy kabuki

Joint production at Kyoto’s Minamiza (2019 - 2021)
Expansion of customer basethrough new productions
Entertainment × ICT

Expansion of theater-going opportunities
through deployment of virtual technologies
Business expansion

Fig. 3. Smart entertainment with Shochiku.

2.1 Smart entertainment
In the field of smart entertainment, we are aiming at
providing a new sports viewing style in collaboration
with J.League, Japan’s professional soccer league.
We are also pursuing the integration of traditional
Japanese performing arts with information and communication technology (ICT) in collaboration with
SHOCHIKU Co., Ltd., a company engaged in motion
pictures and theater operations, including kabuki(Fig. 3).
Kyoto’s Minamiza Theatre, famous for its kabuki
plays, reopened on November 1, 2018, with equipment supporting NTT’s Kirari! immersive telepresence technology. The plan is to co-produce performances at this theater starting in the summer of 2019.
Although the Minamiza Theatre has a long and richtradition, our attempt is to fuse new traditions andnew arts.
2.2 Smart manufacturing
We are undertaking a variety of projects in the fieldof smart manufacturing. For example, we are working on smart plants with JSR Corporation, remotecontrol of construction and mining machinery with
Komatsu Ltd., data usage at manufacturing sites with
FANUC CORPORATION, and detection and response
support for unknown cyber-attacks on industrial control systems with Mitsubishi Heavy Industries, Ltd.
Our project with JSR on smart plants involves acollaboration with Accenture Japan Ltd. on digitizing
explicit knowledge using AI, voice recognition, andsensors (Fig. 4). Going forward, we aim to connect all
NTT Technical Review

Vol. 17 No. 2 Feb. 2019sorts of things through Internet of Things (IoT) technology as a connected value chain and to create astructure that enhances added value not in a partial
manner but throughout the industrial complex. Doingso, however, is an extremely difficult task since batteries cannot be used. A short circuit in a battery can
result in an explosion, which is why there are regulations prohibiting the use of batteries in petrochemicalcomplexes. For this reason, we are using explosionproof sensors developed by NTT Advanced Technology and inQs Co., Ltd. that incorporate dye sensitization technology for generating electric power using
infrared light.
2.3 Smart mobility
In the field of smart mobility, we are working with
Toyota Motor Corporation on fundamental technologies for achieving autonomous driving, with NYKLine and MTI Co., Ltd. on ship safety and environmental initiatives, and with East Japan Railway Company on seamless transportation.
(1) Toyota
Full-scale verification testing with Toyota towardautonomous driving began in December 2018. Up tonow, trials involving the running of one or two cars
have taken place at various locations, but in the
future, a platform that can process data on a scale ofseveral tens of millions of vehicles will be essential.
With this in mind, our initial experiments will assume
the operation of five million vehicles. These experiments will test a connected car platform that canprocess the data collected from cars at a datacenter8

Feature Articles

Digitization of explicit knowledge of skilled workers
Smart industrial complex

Smart plant
Materials
Liquidlevel

Products

AI

Image

Skilledworkers

Connected

Sound

Communication

Voice
recognition

Vibration

Facility

Sensors

Connected

Connected value chain

Inventory

Workers

Utilities

Explosion-proof sensor network
Data aggregation
Digitize explicit knowledgeof skilled workers.

Chemical plant platform

Fig. 4. Smart manufacturing with JSR.

Establishment of fundamental technologies to realize autonomous driving
Toyota• Operation of connected cars

×

NTT Group• Development and verification of 5G,
edge-computing technologies

Development and testing of fundamental technologies
assuming operation of 5 million vehicles* Verification testing began in December 2018
Data collection/
accumulation/
analysis foundation
End-to-end verification
Edge
computing

Datacenter
Large-scale,large-volumedata collection

5G

Real-time
data delivery

IoT
network

Agent
5G: fifth-generation wireless systems

Fig. 5. Smart mobility with Toyota.

and transfer the data processing results to othervehicles in the field to achieve an end-to-end data
flow (Fig. 5). The plan is to continue with these tests
during specific time periods and assuming a varietyof cases. I would like to turn this platform that we are
creating together with Toyota into a model that can beused by other companies throughout the world,
including other automobile companies.
NTT Technical Review

Vol. 17 No. 2 Feb. 2019(2) NYK Line and MTI
We are working with NYK Line and MTI on asmart ship initiative involving an onboard IoT platform that can improve fuel economy, achieve preventive maintenance, and enhance safety by collecting
onboard data and managing that data even from land
using satellite communications (Fig. 6). I would liketo apply this platform to container ships and other9Feature Articles

Evolution of shipping industry safety, environmental initiatives
Demonstration project
to remotely operate a ship

Joint test of
next-generation onboard IoT platform(initiated by Ministry of Land,
Infrastructure, Transport and Tourism)• Studying requirements for remote operating
Satellite
systemcommunications • Testing, creating standards using actual ship
Analysison landUse aboard ship
Data collection

Insurance

Data inverter
Remote distribution
and management of
onboard applications

Data

Advance warning taking accountof left turning of mother ship

Data
distribution
Plannedroute ofmother ship

Mother ship

Source: Ministry of Land, Infrastructure, Transport
and Tourism press release
July 25, 2018

Fuel economy,other improvements

Preventive maintenanceLabor saving, enhanced safety

Environmentally friendly ships

Optimization of ship operation

Autonomous ship navigation

Fig. 6. Smart ships with NYK Line and MTI.

types of vessels with the aim of achieving a ship-toship connected world.
3. NTT Group Medium-Term Management
Strategy: Your Value Partner 2025
In autumn 2018, I announced The NTT Group
Medium-Term Management Strategy called Your
Value Partner 2025. Around that time, the Japanese
government was promoting the separation of handset
and network charges in the mobile communications
market. To remain a market leader, I therefore asked
NTT DOCOMO to announce a cut in mobile phone
fees and incorporated the reduction in revenue in our
Medium-Term Management Strategy.
In retrospect, the separation of terminals also took
place in 1985 when competition was introduced in
the fixed telephone (landline) market, or public
switched telephone network (PSTN). This separationof telephone sets and networks in the PSTN resulted
in a significant expansion of the telephone-set market. Similarly, the separation of mobile handsets
should stimulate competition in the mobile communications and handset markets, which I think is healthy
all in all. On the other hand, this presents a challenging situation for NTT DOCOMO, but by entering intosuch a situation on our own initiative, we can face thisissue with the intention of becoming a company thatcan provide our customers with even better services
while remaining the market leader.
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

3.1 Future trends
The Medium-Term Management Strategy envisions 3-year, 5-year, and 7-year milestones. Here, thefifth-generation wireless system (5G) is scheduled tocommence within three years, and the PSTN isscheduled to complete its migration to Internet protocol (IP) telephony in seven years. At the same time, itappears that globalism and bloc-ism in society willcontinue to coexist for some years. Amid thesetrends, Osaka City was selected to host the World
Expo in 2025. The NTT Group as well is promoting
a variety of initiatives with the aim of supporting theTokyo Olympic and Paralympic Games a year and ahalf from now and the Osaka Expo in seven years.
There are also key trends taking place in the worldof IT. Given the imminent end of Moore’s law, theneed has arisen for new architecture and systems, andmeanwhile, it is predicted that robots will replace halfof the workforce in seven years. To put it another way,
this means that each and every person will be able toengage in more enriching work and cultural activities.
I think that we can expand our market by supporting
people in these endeavors.
3.2 Further phase changes
In regard to phase changes as a communications
and IT operator, NTT’s business model is making atransition from business-to-consumer (B2C) to
B2B2X, as I mentioned earlier. We can also expect
all-out competition as mobile handsets and the10Feature Articles

Supportour customers’
digital transformations.

1. Promote B2B2X model.
2. Roll out 5th-generation wireless systems.
3. Provide personal services.

Accelerate
our own
digital transformation.

4. Enhance competitiveness in global business.
5. Drive self-digital transformation in domestic
business.
6. Migrate PSTN to IP networks.

Leveragetalent, technologies,
and assets.

7. Enhance and globalize R&D.
8. Create new lines of business (real estate etc.).
9. Contribute to vitalization of regional societies andeconomies.
10. Disaster countermeasures
Promote ESG management, and enhance thereturns of shareholders to improve corporate value.

Fig. 7. Medium-Term Management Strategy.

network come to be separated, as well as shifts from
communications services to digital services, from thecloud to virtualization and AI-based infrastructures,and from mobile/FTTH (fiber to the home) to 5G andsmart devices in access networks.
3.3 O utline of Medium-Term Management Strategy
NTT seeks to become Your Value Partner that customers continue to select. To this end, the MediumTerm Management Strategy calls for both growthinvestment and cost reductions to increase profits,
and introduces return on invested capital (ROIC) as a
key performance indicator.
The strategy declares the NTT Group mission to beresolving social issues and the NTT Group vision tobe embodied by the Your Value Partner concept. By
pursuing this mission and vision, we would like tocontribute to achieving a sustainable society by meeting SDGs (Sustainable Development Goals) initiatedby the United Nations and to the realization of Society 5.0 proposed by the Japanese government [3]. I
believe that our mission is to solve social problems
through collaboration with our business partners
while maintaining a healthy balance between theroles of a public utility and private company that the
NTT Group has had from the beginning.
While we are aiming to be Your Value Partner, themain figure of the NTT Group vision is people. In theend, it is people that drive innovation, technology
development, and research and development (R&D),
and the values shared among people are connections,
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

trust, and integrity. The idea of connections in thiscase does not refer to network connections but rather
to the connection of things by IoT and the connectionof systems going forward. It also includes the conceptof helping people to connect with each other and connecting the present with the future.
The NTT Group Medium-Term Management Strategy consists of four elements (Fig. 7). To begin with,
we will support our customers’ digital transformationby accelerating our own digital transformation. At thesame time, we will pursue growth by leveraging talent, technologies, and assets. Finally, at the base of
these elements, we will promote ESG (environmental, social, and governance) management and shareholder return. We have established ten measures toachieve these goals. Here, I would like to highlightthe measures related to R&D.
(1) Roll out 5G services
The 5G system is slated to be the next network.
Given that it is a public network, end-to-end services
will be achieved by adding private networks andfunctions and systems within factories, on vehicles,
and in other areas. The three-layer model shown in
Fig. 8 features overlay solutions at the top, but toprovide such solutions as social systems, various
types of things will have to be coordinated within the
network-services layer. Additionally, 5G aside, an
optical fiber network will run as far as base stations,which means a transport layer at the bottom of thismodel that will also require coordination. In otherwords, it is necessary to deploy functions that canorchestrate and manage all three layers, and these11Feature Articles

Cognitive
Foundation

Service provider operating social systems

API

Network services
API

Ctrl

Provider A

Provider B

API

Multi-orchestrator

Edge
Ctrl
End-to-end coordination of ICT resources

API

Provider C

Ctrl
Agent
Device

Ctrl

Transport

Wholesale

Agile allocation of ICT resources
Optimization of structure

Cloud

Ctrl

Providing social systems

API

RetailOverlay solutions

Ctrl

PublicPrivateClients/Customers

API: application programming interface
Ctrl: controller

Fig. 8. Roll out 5G systems (3-layer model).

“One NTT” global growth strategy
Business modernization

Accelerate IT-as-a-Service
with software-defined technologies.
NTT global procurement company
Global talent

Integrated capabilities

Industry advisory expertise drives
outcome-based solutions.

Disruptive innovation
Global innovation fund

Disruptive Innovation Approach
Research centers (Outside of Japan)

One NTT

Global branding

Fig. 9. Enhance competitiveness in global business.

functions will be achieved through software-definedtechnologies. One example of an overlay solution
here is the Cognitive Foundation platform we areworking on in the City of Las Vegas.
(2) Enhance competitiveness in global businessWe are planning to reorganize the NTT Communications Group and Dimension Data Group into two
companies, one focused on the global market and theother on the Japanese market, by July 2019. As a corporation reorganized under the One NTT brand, I
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

would like to support the modernization of our customers’ businesses (Fig. 9). To this end, we will provide a variety of integrated solutions from datacenter
services to applications, and we will do this throughoutcome-based solutions driven by industry expertise, IT-as-a-Service using software-defined technologies, and efficiency gains via an NTT global procurement company.
However, our customers’ digital transformations
cannot be achieved by this alone. There will be no12

Feature Articles

Leveraging technologies/assets (equipment),
upgrade government/life services, support revitalization of local industry

Universities

Local
Local
Local banks
governments companies

Collaboration with local governments, various partners
Community-based(sales/support)
Group management resources
Latest technologies(AI, RPA, IoT, …)

Equipment/bases(Datacenters/cloud/building)

RPA: robotic process automation

Fig. 10. Contribute to revitalization of regional societies/economies.

appealing solutions without including disruptive
innovation. For this reason, we have establishedglobal R&D centers for long-term development projects and created a global innovation fund. The purpose of this fund is not to launch new businesses but
rather to connect people who wish to pursue new
business ventures and support them through mediumterm incubation. The fund was launched in the UnitedStates with an American as head. Additionally, theDisruptive Innovation Approach team will enhance
our advisory functions. This team will propose adigital or smart world to our customers in the shortterm. In this way, we plan to incorporate long-term,
medium-term, and short-term initiatives within oursolutions.
(3) Enhance and globalize R&D
We will strengthen our joint R&D. I think that communications and IT should be and will be natural
things—if not, they would not be usable. We would
like to conduct more research on natural awareness,
techniques, and methods of use. In addition, I would
like to expand basic research beyond present levels atoverseas bases in such areas as super-secure certification technology, quantum computation theory, andbiomedical technology. To this end, we will increase
our R&D investment in high-growth areas by about
10% to 2 trillion yen over the next five years. Apartfrom this, we will invest 1 trillion yen over the next
five years in 5G infrastructure build-out.
(4)	Contribute to revitalization of regional societies/
economies
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

The Cabinet Office of Japan is advocating a Super
City initiative, while the Ministry of Land, Infrastructure, Transport and Tourism is promoting the conceptof Compact Cities. For Japan, city development is amajor issue going forward. In this regard, I think that
we can support various regions in Japan using the latest technologies, facilities, and sites of the NTT
Group (Fig. 10). Recent years have seen a confrontation between movements in socialism (nationalism,
bloc-ism) and in liberalism (globalism, borderlessworld). While nationalism and bloc-ism are determined by a de jure standard, globalism and the borderless world are determined by a de facto standard.
From here on, however, if we devise the solution ofproblems together with local communities, the concept of glocalism, which combines the meanings ofglobal and local, will no doubt become pronounced insociety. This will result in a structure 70–80% of
which is dominated by local, and in this case, I
believe that glocalism will be determined by a consensus standard (Fig. 11).
(5) Disaster countermeasures
Finally, I would like to talk about disaster countermeasures. In Japan, 2018 was a year of many disasters. Up to now, we have been striving to improve thereliability of the communications network, to secure
critical communications, and to achieve prompt restoration. From here on, we must further reinforce thecommunications infrastructure. This will require proactive disaster responses using AI and other measures. Furthermore, taking into account the growing
13

Feature Articles

VS.

Socialism
Nationalism
Bloc-ism

De jurestandard

Liberalism
Globalism
Borderless

De facto
standard

Plus
LiberalDemocracy
Glocalism

Consensus standard

Fig. 11. Global + Local.

number of international visitors to Japan, we must
work on disseminating helpful information in foreign
languages as well.
3.4 Medium-term targets
The Medium-Term Management Strategy setsmedium-term financial targets that we seek to meetby executing the measures I have just described.
These include a 50% increase in earnings per share(EPS) and cost reductions of at least 800 billion yen
in five years. I would like to achieve this EPS growthby increasing profit. Thus, I want to increase overseas(outside Japan) sales from US$18 billion to $25 billion and raise overseas operating income margin from
3% to 7%. Here, we set ROIC to 8% to aim for an
efficient return on investment.
NTT’s total revenue stands at $106 billion, so its
global revenues are still about 19% of the total. NTT
is ranked 55th on the Fortune Global 500. It provides
network services in over 190 countries and regions
and ranks second in the world in datacenter space.
NTT employs about 300,000 employees worldwide.
4. Accelerate our own digital transformation to
become Your Value Partner
In 1985, NTT’s revenues from PSTN businessaccounted for 83% of total revenues, but today, withNTT Technical Review

Vol. 17 No. 2 Feb. 2019voice revenues from cell phones included, it accounts
for only 18%. In other words, all other sources ofrevenue have expanded substantially. This is a changethat took about 30 years to occur. From here on, I
would like to accelerate our own digital transformation to become Your Value Partner, meaning that ourcustomers continue to choose us over others. As aconsequence of this endeavor, our business portfolio
will undergo great changes. I ask for everyone’s support in determining what the new NTT should
become, and I look forward to promoting businessdevelopment while holding discussions with our customers and other stakeholders.
References
[1]
[2]

[3]

Press release issued by NTT Group, “NTT Group to Launch Smart
Cities Initiative with Dell Technologies,” May 1, 2018.
http://www.ntt.co.jp/news2018/1805e/180502a.html
Press release issued by The State of Nevada, City of Las Vegas, andNTT, “The State of Nevada, City of Las Vegas and NTT Reach an
Agreement to Accelerate Smart City Technologies,” Dec. 7, 2018.
http://www.ntt.co.jp/news2018/1812e/181207a.html
Government of Japan, “The 5th Science and Technology Basic Plan,”
Jan. 2016.
http://www8.cao.go.jp/cstp/english/basic/5thbasicplan.pdf

Trademark notes

All brand names, product names, and company/organization names thatappear in this article are trademarks or registered trademarks of theirrespective owners.

14

Feature Articles: Keynote Speeches at NTT R&D
Forum 2018 Autumn

Making the World Smart andTechnology Natural
Katsuhiko Kawazoe
Senior Vice President, Head of Researchand Development Planning, NTT
Overview

This article introduces NTT’s latest research and development activities
based on a lecture presented by Katsuhiko Kawazoe, NTT Senior Vice
President, Head of Research and Development Planning, at NTT R&D
Forum 2018 Autumn held November 29–30, 2018.
Keywords: smart world, highly realistic communication, artificial intelligence

1. Transformation of the world with B2B2X
Until the present time, the NTT laboratories haveengaged in research and development (R&D) mainly
to support NTT’s services and systems. NTT has
shifted its business strategy to the business-to-business-to-X (B2B2X) model, thus placing greater
emphasis on R&D for value creation with partners.
We have been able to develop groundbreaking coinnovations in collaboration with numerous parties.
For example, in 2014, Mitsubishi Heavy Industries,
Ltd. and NTT agreed to work together on R&D forsocial infrastructure × ICT, which refers to the integration of social infrastructure and information andcommunication technology (ICT). The two companies are engaged in different business fields, but one
technology linked them, and this connection led to a
discovery that transformed the manufacturing industry. From the wide-ranging research results produced
by NTT, Mitsubishi Heavy Industries set its eye on
one technology, one that NTT had never imagined
would catch their attention. They took an interest in
optical fiber technology.
The photonic crystal optical fiber, which was an
NTT first, was characterized by having air holes
inside the fiber, and light propagated through these
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

holes while being confined in them (Fig. 1). Bychanging the hole diameter and the spacing between
the holes, we can finely control the refraction index
of the light in the fiber and achieve optical transmission characterized by extremely high output andunparalleled quality.
Application of this optical fiber technology forcommunication to laser beam machining is the key
objective of the cooperation between the two companies. However, because the light energy level required
for laser beam machining was more than 10,000
times higher than that required for communications,
we faced a number of new challenges. We solved
these problems, and optical fiber technology for communications found a new application in laser beammachines for cutting and welding.
It had not previously been possible to transmit ahigh-power single-mode laser beam for machining
over more than a few meters. The application ofNTT’s technology made it possible to extend the distance by several dozen times. The people at Mitsubishi Heavy Industries discovered new value in our
technology that those at NTT could never have envisaged on their own. This is a prime example of coinnovation through B2B2X. The case involving Mitsu­
bishi Heavy Industries originated from our attempt to15

Feature Articles

Social infrastructure × ICTHigh power laser processing

Photonic crystal optical fiber

Conventional
optical fiber

Photonic crystal optical fiberCore(Light path)

Air hole

Extends transmission distance by several tens of times and makes remote processing possible

Fig. 1. Technology transforming the manufacturing industry.

NATURAL
diverseefficientCreativity

inclusive

ubiquitoussurrounding

adaptable

spontaneous

Imagination

flexible
unawareheart touching

Relationshipcomfortable

Sustainable

Fig. 2. The concept of natural in digital transformation.

solve a problem besetting their business. In the end,
we have produced a result that will dramaticallytransform the manufacturing industry. Looking back,
we realize that this constituted a transformation of theworld.
2. Keyword for making smart world
a reality: natural
NTT has innovative technologies that originated in
its R&D—technologies that have been the best in theworld, the first in the world, and that have amazed theworld. With these technologies and through co-innoNTT Technical Review

Vol. 17 No. 2 Feb. 2019vation with a range of partners, we will push digitaltransformation in society and industry to solve theproblems they face, and thereby make a smart world
a reality. To do this, we will strengthen our involvement in a wide range of technical fields.
How should we advance these technologies andpush digital transformation so that everyone regardless of nationality, age, or background can benefit
from technology? We believe the keyword for thisendeavor is natural (Fig. 2). We are aiming at thecreation of a world in which technology keeps caringeyes on the lives of people from all walks of life without them being aware of it; a world in which technology
16Feature Articles

Acoustic
reproduction

Sporting venue

Collecting and processing information
Multi-angle
video

Extracted
image

Background

Extracted
audio

Synchronized transmission

Virtual image

Background

Acoustic
reproduction

Ultra-realistic presence design
Virtual image
Remote venue

Fig. 3. Ultra-realistic communication that conveys excitement.

sometimes helps people do things more efficiently
and appeals to their feelings; a world in which technology provides an environment that is friendly toboth people and the earth. This initiative will lead toa world in which people can engage in human-centered activities. Let me introduce some of our activities that are aimed at making such a world come true.
3. Ultra-realistic communication thatconveys excitement
The long-awaited regular broadcasting of 4K and
8K television (TV) started on December 1, 2018, inJapan. Its high definition of video images will offer ahigher than ever sense of excitement. As the next step,
we want to realize a world in which people transcendtime and space and experience excitement as if they
were at the site where an event was unfolding. This is
the ultra-realistic communication that NTT is aimingat (Fig. 3).
For example, in the fashion show Tokyo Girls Collection 2018, which was held at the Yokohama Arena
on March 31, 2018, scenes from the show were transmitted to a remote live viewing site in real time,enabling a large number of people to view the scenes
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

happening at the event site. Kirari!’s ultra-wide video
synthesis technology stitched together a number of
4K videos in a natural manner in real time to produce
a video that allowed the audience to enjoy a highsense of reality.
In addition, the videos shown on long verticalscreens, three-dimensional (3D) and virtual reality
videos, and multi-angle videos displayed on tablets
were synchronized precisely using Kirari!’s advanced
media streaming and synchronization technology
called Advanced MMT*. Kirari! is an ultra-realistic
communication technology that enables people at any
location to enjoy the sensation of being at an event
venue. It brings to remote sites the excitement andsensation of tension that usually only those at an
event venue can enjoy. The ability to share in the
excitement of a game or event is what we aim toachieve with Kirari!.
Kirari! has been continuously evolving since the*Advanced MMT: Extended protocol of the MPEG Media Transport (MMT) undergoing standardization in ITU-T (International
Telecommunication Union - Telecommunication Standardization
Sector) Study Group 16 Immersive Live Experience. The MMT
is an optimized protocol for synchronous data transmission developed by the Moving Picture Experts Group (MPEG).

17

Feature Articles

concept behind it was announced in 2015. Initially,
viewers’ attention is captured by its quasi-3D image
display, but what is more important is that Kirari! creates a natural and highly realistic space by decomposing a scene into elements such as the objects’ video
and audio streams, transmitting them separately, andrecomposing them in a way that is best suited to theconditions at the viewing site.
I was once deeply moved by the performance of a
figure skater in a top-level competition. Her entry into
this event was to be the culmination of her longefforts, but she lagged well behind in the short program. Despite that, her performance in the free program the next day was outstanding, gaining her sixth
place in the end. I believe that many people aroundthe world were also moved when they witnessed hercomplete her performance, look up, and start to cry.
Why were we so moved at that time? I believe that wewere impressed not just by her performance but also
because the images of the path she had followed fromchildhood, her aspirations for that contest, and hererrors in the short program ran through our minds likefragments of a story and were multiplied by thevideo.
Kirari! will evolve further with the aim of not onlytransmitting the video and audio faithfully but also
evoking deep emotions in the hearts and minds ofviewers based on a relevant story, such as past experiences and knowledge. Ultra-realistic communicationthat conveys deeply felt emotions. That is the naturalness that we aim at.
4. Artificial intelligence (AI) that guesses
what you think
Technologies for listening and speaking AIs are
now being incorporated into home appliances and arecommonly used in our everyday lives. If we are toconvey human feelings in a natural manner withoutcausing the person involved to be conscious of interacting with a machine, it is essential to make further
advances in speech dialogue technology. For example,
if AI can handle unstructured chatting, which makes
up a good part of our conversation, it can communicate with people naturally. Therefore, research on
chat technology is underway at the NTT laboratories.
We are developing an AI system that can converse
more naturally by giving it a distinct character and,
consequently, friendliness. In order to initiate andsustain a pleasant conversation, we have developed afunction that responds with not just words but also
through gestures such as nodding in agreement, a
NTT Technical Review

Vol. 17 No. 2 Feb. 2019verbal function whereby appropriate responses are
made that indicate agreement with what you say or
think, and a function whereby a follow-up question ora question relating to what you have said is asked. We
have incorporated these into Totto, the android of Ms.
Tetsuko Kuroyanagi, Japanese actress and TV presenter [1].
Viewing AI that recognizes objects in the surroundings and the current situation will also become morenatural. NTT has been researching robust image recognition technologies. They include, for example,
robust media search technology that requires only a
few reference images in order to determine that an
object in question is identical to the object in the reference images, and change point detection technologythat instantly locates change points in observational
photos taken from a satellite.
In addition, we have developed angle-free rigid andnon-rigid object recognition technology that is
expected to advance image recognition technology
dramatically (Fig. 4). Things that AI needs to recognize are not necessarily limited to rigid objects. This
new recognition technology can tell that an object
with a shape that can change, such as a product in abag, is identical to the object in the reference image.
This ability is expected to significantly expand theapplication areas of image recognition AI.
The performance of AIs when listening, speaking,and viewing things is improving. The time will comewhen it is more than adequate. We believe that atsome stage in the future, thinking AI that supports
human thought processes and leads to co-creation bypeople and AI will become more important than ever(Fig. 5). The current technology for Totto is still at astage where the distinct character of Ms. Kuroyanagi
has been incorporated into the robot. In the future, AI
needs to incorporate her values and personality. The
AI we are aiming for at NTT is one that incorporates
various values and personality traits and helps people
to consider complicated problems for which there isno single answer.
In the future, we will aim for AI that, while conforming to commonly accepted rules and morals,
absorbs all of the perceptions of value that may ariseout of geographical regions or customs. Such AI
could be called generous AI or sincere AI.
5. Stress-free device
When we think of the next personal communication
device that will go beyond the evolution of cell
phones and smartphones, we think that what is needed18Feature Articles

Recognition result:object nameQuery image

Matching between queryand reference image

Reference image

Fig. 4. Angle-free rigid and non-rigid object recognition technology.

Differences insenses of value

Real-world data

Thinking AI

Generous AI

Listening AI

Speaking AI

Viewing AI

Fig. 5. AI that guesses what you think.

is a terminal or device that frees people from being
dominated by applications and settings.
CUzo is a device that can be operated very naturally just by pointing it at something such as a person
or a landmark. You can get information about something just by pointing the device at it. For example,
CUzo will present information about a tourist spot in
the user’s own language on its transparent display,provide easy-to-understand navigational advice in an
unfamiliar street, or enable people from different
countries to have a face-to-face conversation by looking at translations that appear on the display (Fig. 6).
We want to enable people to have a natural experience by doing away with annoying tasks such asactivating and operating applications, and having thedevice provide services in response to prompts conveyed by an individual’s natural actions.
We have developed this system in collaboration
NTT Technical Review

Vol. 17 No. 2 Feb. 2019with Panasonic Corporation. It uses device function
virtualization technology developed by NTT, which
virtualizes the processing functions conventionallyperformed within a device such as a smartphone, andplaces them in a cloud or on an edge computer. This
technology makes it possible to provide advanced
services even with simple devices.
After we have made it possible to use devices without the conscious effort of setting up and operating
applications, what will come next? We foresee afuture in which we no longer need to be conscious ofvisible devices, and various things around us lookafter our lives. For example, several ICT devices in aroom work together and provide an illusion that araincoat hung on the wall appears to be trembling andthe floor looks wet, thereby letting the resident know
in this natural manner that it is going to rain today(Fig. 7). This is one of the manifestations of naturalness.
19

Feature Articles

Transmissive display device

Device function virtualization technology

Fig. 6. Stress-free device.

To pursue this concept, we plan to launch a new project called Point of Atmosphere. Please follow what
we do in this field in the coming years.
6. Future network that blends seamlessly
into society

Fig. 7. A world where we no longer need to be consciousof visible devices.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Networks support various services. They will also
evolve to become more natural.
A next-generation network will integrate various
functions and roles across different layers. It will
understand social order and priority and operate in anatural manner without humans being required tomake conscious efforts to select the optimal choice. A
mechanism we are working on for this purpose is the
Cognitive Foundation®. It manages and operates
various resources in different layers in an integrated
manner. In the public safety solution in the City of
Las Vegas, the Cognitive Foundation manages various ICT resources such as cameras, sensors, edge
computers, networks, and clouds, as necessary in an
optimal and integrated manner in order to flexibly
adapt video monitoring resources to what is needed ata particular time.
At the current stage, the Cognitive Foundation only
performs basic operations automatically, but our aim
is that as the network AI continues to learn, it will
execute globally optimal control in real time in a constantly changing environment and achieve advanced
coordinated operations. The Cognitive Foundation
will expand its coverage area to people, towns, transport and energy, work with enterprises involved invarious layers of society and applications, and20

Feature Articles

Phase sensitive
amplifierProblem setting unit
(PSU)

Fig. 8. Computer that solves challenging problems with light.

orchestrate everything from services to devices,
thereby achieving optimization of the entire society
in a natural manner. For this purpose, we will work on
scalable data processing infrastructure and supersecure authentication infrastructure technologies.
7. Computer that solves challenging problems
with light
If we are to enable people to lead a better life in anatural manner, we must boost the power of computers. To overcome the limitations of conventional von
Neumann computers, the NTT laboratories are developing LASOLV, a computer that solves challenging
problems with light (Fig. 8). Conventional digitalcomputers solve a problem by turning it into a mathematical problem. LASOLV solves a problem not asa mathematical problem but as a physical problem.
This has made it possible to solve hard problems that
could not be cracked by conventional digital computers.
LASOLV is implemented using leading-edge optical communication devices and optical parametricoscillator pulses generated by phase sensitive amplifiers, both of which have been developed by the NTT
laboratories over many years. A series of opticalpulses flow in a 1-km-long optical fiber ring. LASOLV
expresses inter-pulse interactions using an opticalparametric oscillator and a problem-setting unit. Thishas made it possible to solve the problems associatedwith optimizing large-scale combinatorial models.
To date, LASOLV has been able to solve relatively
simple grouping problems. It has now become possible for LASOLV to expand its targets to a wider
variety of problems such as the Japanese map coloring problem and scheduling problems. For example,
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

assume that the possibility for a town consisting ofresidential, commercial, industrial, and green sections to grow to become a better town is predicated on
the condition that sections belonging to the same category should not exist side by side. Let us consider asimulation of town development that meets this condition. The beauty of LASOLV is that it can solve a
problem incredibly quickly that conventional digitalcomputers cannot solve in a short time.
We are now considerably expanding the libraries of
LASOLV software and advancing its software development environment so that not only experts but also
general programmers can make use of it.
In addition, we will increase the number of bits in
hardware from 2048 to 100,000 so that LASOLV canbe applied to a wider variety of fields. One example
is drug development. At the basic research stage ofsearching for chemical compounds that can be usedtherapeutically, LASOLV has the capacity to quickly
identify good combinations of chemical compounds
so that new drugs can be developed in a short time. To
solve traffic jams and city planning, LASOLV will
suggest a better route by calculating a recommendedroute from one point to another or the most efficientroute when it is necessary to visit several places. In
the field of AI, LASOLV will have the ability to findfrom a data set to be used in machine learning, asample that is the closest to the data being sought.
This will lead to more robust learning.
8. Activities to accelerate innovation
I have so far introduced our R&D results and the
direction we are taking. Let me reveal the details of
our new measures designed to accelerate further
innovation. The underlying policy is globalization of21

Feature Articles

NTT laboratories

NTT

Enhance research of
cutting-edge technologies.

Global holding company(NTT, Inc.)
Global OpCo
Japan OpCo

July 2019
Establish NTT Research, Inc.
in Silicon Valley andexpand it globally.

NTT DATA
Global procurement company(NTT Global Sourcing)Global innovation fund(NTT Venture Capital)
Disruptive Innovation Approach
NTT Research, Inc.

NTT  Laboratories
NTT CIS Laboratories

Datacenter asset firm

NTT MEI Laboratories

Fig. 9. Establishment of research organizations outside Japan.

our R&D. We are taking the following three measuresto achieve this:The first is global utilization of R&D results. We
will take the R&D results of the NTT laboratories
outside Japan and deploy them in ways adapted toindividual regions.
The second is globalization of research targets. We
will strengthen R&D that is adapted to global needs.
The third is establishment of research organizations
outside Japan (Fig. 9). There is apprehension todaythat basic science and technology, the very source ofinnovation, are declining in Japan. We believe thatbasic research that serves as the foundation supporting the activities I have outlined above is the key thatwill allow us to take the next leap forward. To expandand strengthen basic research, we will establish abasic research organization overseas. It will be called
NTT Research, Inc.
Three laboratories will be established within NTT
Research, Inc. The first is NTT Φ Laboratories. Thisfacility’s mission is to make new discoveries anddevelop new technologies that will dramaticallytransform the world in the area of quantum scienceand computing, which is a co-creation area encompassing both physics and informatics. We came up
with the idea of using PHI or Φ from the Greek alphabet—as an acronym of physics and informatics—for
NTT Technical Review

Vol. 17 No. 2 Feb. 2019the name of the laboratories. The focus at Φ Labs will
be on basic research in the field of quantum theory,
which will lead to future quantum computing technology and the formulation of completely new theories, including the application of quantum theory to
information processing. We will invite Professor
Yoshihisa Yamamoto, professor emeritus of the
National Institute of Informatics and of Stanford University and the project manager of ImPACT (Impulsing
Paradigm Change through Disruptive Technologies
Program), which has collaborated with us in thedevelopment of LASOLV, to become the director ofΦ Labs.
The second is NTT CIS Laboratories. This organization will work on advanced cryptographic theory
and encryption, and information theory, which is thebasic theory underlying secure information exchange
in a complicated distributed environment. The director of CIS Labs will be Dr. Tatsuaki Okamoto, NTT
Fellow and last year’s recipient of the RSA Conference Award, which is one of the most prestigious
awards in the area of cryptography.
The third is NTT MEI Laboratories. They will be
engaged in medical and health information processing,which will assist the establishment of a natural relationship between people and ICT. To head MEI Labs,
we will invite Dr. Hitonobu Tomoike, medical doctor
22

Feature Articles

and advisor to the Sakakibara Heart Institute. He has
considerable influence both in Japan and abroad as aphysician expert in the cardiocirculatory system andis also well-versed in ICT.
The key aspect of these appointments is that all
three individuals are active globally in their respective fields of expertise and thus have wide-ranging
international human networks.
We will first establish NTT Research, Inc. in Silicon Valley, California, where research in these fields
is advancing, and then extend its geographical presence to other parts of the world. Through R&D at
global sites focusing on basic research, and in collaboration with universities, both at home and abroad,and with other partners, we will aim for the summitof basic research and produce unparalleled research
results that will bring about game-changing innovations.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

9. Future outlook
The NTT laboratories have shown the potential of
new technologies well in advance of their rivals. We
will aim to achieve impactful technical innovation,
keep abreast of the changing times, and open up thenext frontier.
NTT R&D will look ahead and paint a picture ofwhat lies beyond the near future. With a view tobringing about a future characterized by naturalness
and to realizing a smart world, we will continue topursue research that will transform the world.
Reference[1] Totto’s room (in Japanese), http://totto-android.com/

Trademark notes

All brand names, product names, and company/organization names thatappear in this article are trademarks or registered trademarks of theirrespective owners.

23

Feature Articles: Value Creation by Leveraging Stateof-the-art Research Results

Ensuring Greater Safety for Our
Firefighters and Our Communities:
Integrating FLAIM TrainerTM andhitoeTM
Tamir Levin, Simon Chessum, James Mullins,
Nobutomo Yoshihashi, and Katsuyoshi Hayashi
Abstract

FLAIM TrainerTM is an immersive, virtual reality (VR) firefighter training simulator with biosensing
and real-time scenario analytics. It was developed in Australia through research at Deakin University andincorporates a patented digital biosensing fabric developed by NTT called hitoeTM. The hitoe material
provides real-time data readings of biomedical signals, for example, electrocardiogram or electromyogram signals. FLAIM Trainer revolutionizes the way emergency services train and prepare to fight fires.
The immersive VR, haptics feedback, and breathing simulation provide firefighters with the ability to
train for dangerous scenarios that are hard to reproduce in reality, expensive, and environmentally harmful.
Keywords: wearable sensor, electrocardiogram, biomedical signal monitoring, haptics, immersive virtual reality, firefighting, emergency services

1. Introduction
NTT subsidiary Dimension Data and FLAIM Systems Pty Ltd., a start-up wholly owned by Deakin
University (Melbourne, Australia), have collaborated
on the integration of biometric data capture and feedback as part of an immersive virtual reality (VR)firefighting simulator called FLAIM TrainerTM. Thishas created an enhanced training experience andimproved the health and safety of trainees and ultimately their fitness to fight in emergency situations.
Under a co-innovation agreement, Deakin University
and Dimension Data have integrated NTT’s hitoeTM
material (Fig. 1) [1] into FLAIM Trainer. The hitoe
biosensing nanofiber material is worn as a vest by thetrainee, which tracks electrocardiogram (ECG) readings and transmits the data in real time for analysis of
firefighters’ heart rate, stress level, and fatigue during
NTT Technical Review

Vol. 17 No. 2 Feb. 2019training. The FLAIM Trainer system includes an
HTC VIVE VR headset, personal protective clothingwith heat generation components, a breathing apparatus simulator, and a patented haptics feedback hosesystem that provides a unique training experience.
With the complementary layer of hitoe, instructors
can monitor the trainee’s vital signs as well as physiological response and performance during training.
Dimension Data’s Cloud Analytics Platform is used
to aggregate the hitoe data with the data captured
from the FLAIM Trainer system during training. The
data is then displayed alongside the visuals of the
training scenario being undertaken (Fig. 2). As thecaptured data and training session are recorded andstored in a cloud environment or integrated into alearning management system, results can be benchmarked, and instructors can review and track performance over time.
24Feature Articles

Fig. 1. hitoeTM smart shirt and transmitter.

Fig. 2. Screen for instructors and trainers.

This innovation is unparalleled in the world of VRbased training systems and will have a much broader
application in the future for other industry uses. The
system not only captures, integrates, and presents live
feedback from all devices and sensors, including
hitoe, but also enables the trainee and instructor to see
how a set of actions (firefighting technique, hose andwater or foam use, oxygen consumption and any
simulation or scenario-related activity) correlateswith and affects performance via heart rate and stress.
This will lead to new insights into how we fight fires
and better manage the health and safety of our emergency workers.
The FLAIM Trainer solution was announced and
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

demonstrated at the launch of Dimension Data’s first
Client Innovation Centre (CIC) in Sydney in August
2018. The CIC showcases a number of technologies
and solutions developed and co-innovated with the
NTT Group and the wider Australian business, government, and academic community [2].
2. FLAIM Trainer design
FLAIM Trainer is a highly configurable and mobile
training system that can be easily transported and set
up at different locations. The hardware (Fig. 3) consists of:•	Industry-standard personal protective helmet and25

Feature Articles

Industry-standard
personal protective
helmetFace mask andbreathing apparatus
systemTask Force
Tip G-Forcebranch/nozzle

HTC VIVE
VR display

HTC VIVE
TrackerProtective
clothing withheat generation
components
Standard hose
attached tomobile hose-line
reel unit

Fig. 3. FLAIM TrainerTM hardware.

clothing with heat generation components;•	An NTT hitoe vest and transmitter that tracks
heart rate, heat stress, and ECG and transmits
data via bluetooth;•	A head-mounted HTC VIVE VR display integrated into a full-face cover, and a customized
oxygen tank that replicates the breathing apparatus used by firefighters;•	A mobile hose-line reel unit and Task Force Tip
G-Force branch/nozzle that can be customized
with haptic feedback, generating up to 300 N offorce;•	A VIVE Tracker mounted on the tip of the nozzle
and two VIVE Base Stations; and•	Support equipment including an operator control
tablet, charging systems, ancillary support tools,
and storage and transport cases.
FLAIM Trainer’s VR simulation produces realistic
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

renders of smoke, fire, water, and foam effects andprecisely matches the physical nozzle settings andperformance, with five detent flow positions from an
optimal straight stream to a fog pattern displayed in
the headset for the trainee and on screen for the
instructor. The nozzle’s selectable 100/200/300/400/
500 l/min @ 500 kPa (5 bar) water pressure control,
combined with the flow type, is reflected in thephysical force feedback for the trainee created by thehaptics system.
The mask and breathing apparatus system, worn asan oxygen tank on the trainee’s back, contains all of
the electronics and processing to capture and measure
the respiration rate of the trainee, the amount of oxygen consumed and whether or not they are communicating effectively during the exercise.
The heat suit realistically replicates the heat thatwould be felt by the trainee in the simulation scenario, including the direct or radiant heat from the fire
itself. The heating elements are controlled via the
FLAIM Systems software that determines the proximity and orientation to the fire, and other heating
effects, and how that affects the individual.
From a software and systems perspective, there are
five standard scenarios available, with additional andcustomized scenarios available via subscription tousers. These base scenarios and skills training coverhose use, compartment (kitchen) fire, aircraft (foam),gas cylinder cooling (BLEVE: boiling liquid expanding vapor explosion), and vehicle and size-up fires(Fig. 4). In each scenario, data feeds from the various
components of the VR simulation and hitoe vest arecaptured and presented on screen for the instructor inreal time. The data is also captured in Dimension
Data’s Cloud Analytics Platform, with data sets
stored and analyzed to provide measurement against
benchmarks, that is, to determine best practices, andfor training results to be logged in each trainee’s
training record, either in a stand-alone application or
the organization’s own learning management systemor other professional development or training system.
Individual or group feedback and recommendations
can then be provided by the instructor based on thereports generated (Fig. 5).
FLAIM Systems has a team of roboticists, modelers, software developers, machinists, fabricators, andengineers who can develop customized solutions andscenarios for specific training exercises and requirements.

26

Feature Articles

Fig. 4. Rendered image of standard scenarios.

FLAIM Trainer VR simulator
NTT hitoe biometric vest
All firefighter trainees wear
hitoe vests and sensors.

Trainee feedback andrecommendations

VR feed and sensor
data capture

Data sets are stored and analyzed
to provide measurement against
benchmarks to determine ‘best practices.’

Local simulation
visualization and real-timemonitoring

Uploads captured data to theDimension Data Cloud
Analytics Platform

Fig. 5. FLAIM Trainer - How it Works.

3. Stress calculations from hitoe
The calculations of the trainee’s level of physiological stress are completed on the hitoe device andreported to the FLAIM Trainer application as a valueof 0 – ~2.3. The FLAIM Trainer software then converts this to a six-bar visual stress indicator, displayed
as part of the instructor’s dashboard (Fig. 6).
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

The stress estimation logic calculates the ratio of
high- and low-frequency oscillations (LF/HF) values
as one of the feature values employing inter-beat(RR) interval values (RRI) representing the fluctuation of heartbeats. LF: total power of 0.04–0.15 Hz
when performing spectral analysis on RRIs; HF: total
power of 0.15–0.40 Hz when performing spectral
analysis on RRIs; LF/HF: increases when sympathetic
27

Feature Articles

Fig. 6. Fit to fight analysis.

nerve activity is activated.

Graeme Bacon, Royal Australian Navy [4].

4. Cost and difficulty of traditional training

5. Reduced environmental impact

One of the key challenges that FLAIM Trainer
addresses is the cost and difficulty of training, which
limit the scale and frequency of firefighting training.
Those costs include establishing and maintaining
purpose-built training facilities, personnel, and travel
time and costs for trainees to attend training sessions.
For the more complex or extreme scenarios, for example, an aircraft fire, setting these up in a real training
facility are virtually impossible or prohibitively
expensive. FLAIM Trainer also provides an opportunity for the community and potential recruits to experience what it is like to fight a fire—all at a low cost, ina quick timeframe, and at any possible location.
“The Virtual Reality roadshow is underway and ismore than likely coming to a location near you! This
Virtual Reality tool will help volunteers train for incidents that they may not see on a regular basis.”—Country Fire Authority (CFA) News & Media [3].
Training members of the world’s navies, forinstance, has historically been difficult—logisticalconstraints mean that naval members cannot fully
train at sea, as they are constrained in the use of training smoke or the use of fire hoses in the middle of avessel. Using industry standard equipment and realworld systems in a virtual environment such as with
FLAIM Trainer removes those constraints and allows
users to train more, train better, and train anywhere.
“A big push for the future will be to get Sea Training Group to take FLAIM Trainer to sea, so Navymembers get the chance to practice with a greaterlevel of realism in their ships.”—Commander

By complementing physical training with a fully
immersive set of virtual scenarios, FLAIM Trainer
can reduce the environmental damage caused by reallife training. It also enables training for scenarios thatare no longer possible to carry out due to environmental, community, or regulatory constraints. Traditional firefighter training can cause extensive environmental harm. In a country such as Australia,
where many areas are suffering from water shortages
and drought conditions, high volume water usage fortraining purposes can be problematic in that it takes
precious and limited water resources away from moreessential needs.
By-products of traditional training activities suchas smoke and other pollutants released through theprocess of burning substances can have a damaging
effect on the surrounding environment and air quality
and also contribute to atmospheric damage. Apartfrom water, the long-term effect of other substancessuch as per- and polyfluoroalkyl substances (PFAS)
used in firefighting foams, has caused contaminationof the surrounding areas and health side-effects for
populations around the world. The Australian government reported that there are a number of specific sites
across Australia where run-off from the historical use
of PFAS-containing firefighting foams has resulted inincreased levels of PFAS in surrounding soil andwater. As an indication of the seriousness of thisissue, the Australian Government has set up a dedicated website to provide the community with more
information on PFAS contamination [5].

NTT Technical Review

Vol. 17 No. 2 Feb. 201928

Feature Articles

Many jurisdictions around the world have bannedor are phasing out the use of firefighting foams containing PFAS chemicals. A number of physical training sites have been closed down due to PFAS contamination across Australia, increasing the cost andavailability of firefighter training, and also limiting
the amount of training that can be conducted safely
with firefighting foams. Furthermore, the potential
health impacts associated with exposure are being
investigated by an Expert Health Panel for PFAS
established to advise the Australian Government [6].
6. Trainee health and safety
The use of firefighting foams in training exercises
also has a direct impact on the trainees themselves.
When these training exercises are replaced with virtual scenarios, there is less exposure over the longterm for firefighters to the potentially damaging
effects of the chemicals used in firefighting foams.
For example, in 2013, Airservices Australia tested
150 firefighters for exposure to toxic chemicals andfound that some firefighters had levels of PFOS
(polyfluoroalkyl substances)—a type of PFAS chemical—from 10 to nearly 20 times higher than thegeneral population [7].
However, firefighting training in and of itself isinherently dangerous to the health of trainees. With
the biosensing data supplied by hitoe, instructors areable to monitor the trainee’s health and safety during
the training exercise, alerting them to immediate risks
to the trainee’s wellbeing or to underlying healthissues that can then be treated to avoid potential harmto the person in future training scenarios or in thefield.
Training-related injuries and fatalities are a serious
concern in the emergency services sector. For
instance, the United States Fire Administration(USFA) reported that approximately 11% (141 out of1305) of the line-of-duty deaths that occurred from2001 to 2013 were training-related. The leading
cause of training-related deaths was heart attacks(50%) followed by traumatic injury (31%). The
remaining 19% were other types of cardiovasculardisease and other diverse circumstances [8].
During 2001 to 2013, the National Institute for
Occupational Safety and Health (NIOSH) investigated 77 training-related fatalities through the FireFighter Fatality Investigation and Prevention Program [8] and reported that of those fatalities, 62(80%) were cardiac-related and 11 (14%) were trauma-related. The investigations included 38 deaths
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

due to physical fitness activities, 23 deaths due toapparatus/equipment drills, 10 deaths due to liveburn exercises, and 5 deaths due to other training
associated circumstances.
In 2017, the situation in the United States had only
improved marginally. The National Fire Protection
Association reported that ten deaths had occurred
during training activities. Sudden cardiac deathclaimed the lives of seven of the firefighters. Three of
those seven were engaged in physical fitness training;
two were involved in search and rescue training; onewas training on vehicle extrication, and one wasinvolved in hose training. Two of the other three training deaths resulted from traumatic injuries. One ofthose firefighters fell from an aerial ladder during
above ground fire training. Another was involved in a
motor vehicle crash while traveling to an off-site drill.
One firefighter died shortly after developing complications from a recent medical procedure while he wasattending a refresher class at the fire station [9].
7. Societal benefits
With firefighters facing long periods of inactivity
and fighting fewer fires than ever before, the coinnovation initiative between Dimension Data, Deakin University, NTT, and FLAIM Systems will significantly boost the fitness and effectiveness of firefighters all around the world and put them in the best
possible position to stay safe and protect our communities. The combination of FLAIM Trainer andhitoe has proven to be a safe, low cost and mobile
solution that can simulate a range of fire events andconditions, enabling firefighters to train more easily
and more frequently. This will make our firefightersbetter prepared and better able to deal with a wider
range of emergency situations and fire scenarios thatthey might face in the real world, which will ultimately result in better community safety. At the sametime, with virtual training supplementing real-lifetraining, there are better health and safety outcomesfor our firefighters, and a reduced impact both on ourenvironment and on the health and welfare of ourcommunities.
CFA News and Media reported that FLAIM Trainer
could be used to engage the community and let themfeel what it was like to be a firefighter in a fire scenario [10]. Steve Warrington, Chief Officer of the
CFA in Victoria stated in a news broadcast that thetechnology enables firefighters to monitor their own
health and wellbeing and to find out how they willreact in the heat [11].
29Feature Articles

8. Future development
FLAIM Trainer is currently commercially available
in Australia and the United States, and there are plans
to launch the product into other worldwide markets.
FLAIM Trainer can be used in fire departments anddefense forces, firefighter training schools, and othertraining providers for emergency services. The innovations demonstrated with FLAIM Trainer and hitoe
have applications across a range of different industries and professions where health and safety is an
issue, and where effective training and emergency
response readiness are important.
The use of technology applied to the challenge oftraining firefighters to be fit to fight also creates aplatform for the ongoing enhancement of current VR
simulations and the development of future training
scenarios.
References
[1]

[2]

K. Takagahara, K. Ono, N. Oda, and T. Teshigawara, “‘hitoe’—A
Wearable Sensor Developed through Cross-industrial Collaboration,”
NTT Technical Review, Vol. 12, No. 9, 2014.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201409ra1.html
Press release issued by Dimension Data, “Dimension Data Launches
First Client Innovation Centre to Bring Cutting Edge Tech Down
Under,” August 1, 2018.
https://www.dimensiondata.com/en/news/dimension-data-launchesfirst-client-innovation-centre-to-bring-cutting-edge-tech-downunder.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019[3]

Country Fire Authority (CFA) News & Media, “Virtual Reality Touring the State,” July 25, 2018.
https://news.cfa.vic.gov.au/-/virtual-reality-touring-the-state[4] Navy Daily, “Simulation Enhances Training Safety,” August 20, 2017.
http://news.navy.gov.au/en/Aug2017/Fleet/3978/Simulation-enhancestraining-safety.htm[5] Website providing information on PFAS by the Australian Government, https://www.pfas.gov.au/about[6] Website of the Department of Health, Australian Government,“Expert Health Panel for PFAS Report,”
http://www.health.gov.au/internet/main/publishing.nsf/Content/ohppfas-expert-panel.htm[7] ABC News, “PFAS Levels Up to 20 Times Higher in Aviation Firefighters, Documents Reveal,” July 31, 2018.
http://www.abc.net.au/news/2018-07-31/pfas-levels-high-in-aviationfirefighters-documents-reveal/10052660[8] National Institute for Occupational Safety and Health (NIOSH), “Preventing Deaths and Injuries of Fire Fighters during Training Exercises,” 2015.
https://www.fdsoa.org/preventing-deaths-injuries-fire-fighters-trainingexercises/[9] R. F. Fahy, P. R. LeBlanc, and J. L. Molis, “Firefighter Fatalities in the
United States-2017,” National Fire Protection Association, Research,2018.
https://www.nfpa.org/News-and-Research/Data-research-and-tools/
Emergency-Responders/Firefighter-fatalities-in-the-United-States[10] A. Stockton, “Real Training in a Virtual World,” Country Fire Authority (CFA) News & Media, February 20, 2018.
https://news.cfa.vic.gov.au/-/real-training-in-a-virtual-wor-1[11] News footage of “Nine News Sydney” (television broadcast), 6pm,
August 1, 2018, Channel 9 (only available in Australia).
https://www.9now.com.au/nine-news-sydney/2018

Trademark notes

All brand names, product names, and company/organization names thatappear in this article are trademarks or registered trademarks of theirrespective owners.

30

Feature Articles

Tamir Levin

Lead Technologist, Strategy & Innovation,
CTO Office, Dimension Data Australia.
He received a double major B.Sc. in computerscience and information systems managementfrom Tel Aviv University in 1994. He joined theCTO Office of Dimension Data Australia in
2015, where he leads co-innovation projects andhelps drive pervasive changes via transformativetechnology and thought-leadership with key clients and partners. Before joining Dimension
Data, he held several technology leadership positions as chief technology officer and chief information officer in progressive organizations in the
USA and Australia.

Simon Chessum

Digital Solutions Practice Manager, Dimension Data Australia.
He received a Bachelor of Computer Systems
Engineering and has been engaged in the development and production of software solutions forover 20 years. He is responsible for development,architecture, delivery management, and practicemanagement across the Victoria and Queensland
businesses. He is currently responsible for managing a team of software development professionals in the field of integration, web, mobile
and Internet of Things systems. His softwaredevelopment interests include a focus on iOS
development and augmented reality solutions.

James Mullins

Nobutomo Yoshihashi

Senior Manager, Research and Development
Planning Department, NTT.
He received an M.E. from Keio University,
Kanagawa, in 1994. He joined NTT in 1994 andmoved to NTT Communications in 1999. Heworked in system integration and project management for corporate customers. He has been in
his current position since 2017 and is responsible
for strategic business creation of hitoe.

Katsuyoshi Hayashi

Senior Research Engineer, Device Technology
Laboratories and Research and Development
Planning Department, NTT.
He received a B.E., M.E., and Ph.D. in appliedchemistry from Waseda University, Tokyo, in
1996, 1998, and 2005. He joined NTT Basic
Research Laboratories in 1998 and began
researching electrochemical biosensing devices
to detect biomolecules, including neurotransmitters and hormones in the brain and blood. During
2007–2008, he was a visiting scientist in the
Biomedical Engineering Department, Universityof Wisconsin-Madison, Wisconsin, USA, where
he studied cell biology with microfluidic devices.
In 2017, he began working with various stakeholders to create new value using NTT R&D
products.

CTO, FLAIM Systems Pty Ltd.
He has an engineering degree with a major inrobotics and a Ph.D. in medical simulation. He
has worked in the robotics and technology sectors for over 20 years and has a strong focus oncommercializing cutting edge technologies into
domains including defense, medical, fire, andmanufacturing. FLAIM Systems is a start-up
from Deakin University delivering next generation training opportunities for first responders.
He has 25 years of experience as a CFA volunteer
and is currently serving as the 1st Lieutenant at
the Grovedale chapter outside Melbourne, Australia.

NTT Technical Review

Vol. 17 No. 2 Feb. 201931

Feature Articles: Value Creation by Leveraging Stateof-the-art Research Results

Utilization of NTT’s R&D Technology
in the Financial Industry
Rika Miyahara, Fumi Enomoto, Yoshihisa Koike,
Seiichiro Kitamaki, Takashi Miyatake,and Takashi Tominaga
Abstract

The financial industry is currently going through major transformations owing to a technologicalinnovation called fintech (i.e., finance and technology). The NTT laboratories have been researching anddeveloping a number of technologies that contribute to the promotion of fintech such as concealment
technology for blockchain, artificial intelligence technology for automated responses to customers, androbotic process automation technology for improved operational efficiency, and we have been able tointroduce these technologies via various NTT Group companies. This article introduces some collaborations with Mitsubishi UFJ Financial Group, Inc., in cooperation with NTT Communications, that wehave been implementing since fiscal year 2017. The future utilization of NTT technology in the field offintech is also discussed.
Keywords: fintech, digitization, teleconference

1. BackgroundIn the financial industry, the development of digitalfinancial technology called fintech has been greatly
transforming financial operations. Technologicalinnovations are being made across a wide range ofapplications; some examples are credit screening byartificial intelligence (AI), automated responses tocustomer inquiries by chatbot, and international
remittances with cryptocurrency using a new information infrastructure technology called blockchain.
NTT and Mitsubishi UFJ Financial Group, Inc.
(MUFG) have been jointly studying digitizationtransformation for about a year, in cooperation withNTT Communications. By introducing various technologies developed by the NTT laboratories andjointly investigating the possibility of utilizing thosetechnologies in the banking business, we are aiming
at mutually accumulating knowledge for creating
business by utilizing our research results in the future.
The technological innovation achieved throughNTT’s research and development (R&D) that attractNTT Technical Review

Vol. 17 No. 2 Feb. 2019ed the interest of MUFG is our immersive telepresence technology called Kirari!. Kirari! comprises anarray of technologies that transmit high-reality images in a manner that makes viewers feel like they wereseeing the objects of the images in person even when
the objects are in remote locations. We have beendeveloping Kirari! with the aim of applying it to relay
broadcasts of entertainment such as sports, kabuki
plays (traditional Japanese performing art), and concerts. When we introduced this technology to MUFG,
we received proposals to apply it for handling situations such as external events, conferences, and customer visits to branches. We subsequently began ajoint study in a wide range of areas on the possibilityof utilizing the technologies of the NTT laboratories
in order to provide new banking services at MUFG.
Currently, to improve management efficiency,
megabanks and other banking institutions are aiming
to increase the number of next-generation branches atwhich customers handle various procedures themselves and to offer new banking services by which
various procedures are completed in customers’
32Feature Articles

Business innovation through digitaltransformation

Technological development and over 40
years of research on speech andtransmission technology

Labor-saving and automation of work
are essential.

Collective strength of the NTT Group andpartner companies

Providing integrated financial services
seamlessly anytime, anywhere according
to customer needs

New video experience (for 2020)

Improving customer loyalty by providing
new experiences

Communication engine and tools

Fig. 1. Joint study on realizing a future vision of banking services.

homes. Using ordinary video calls to customers who
visit branches is one approach; however, communication via ordinary video calls might be perceived ascold and unwelcoming by customers. MUFG thoughtthat applying NTT’s advanced technologies such as
Kirari! and corevo® AI technology would make itpossible to respond to customers from a remote location while maintaining warmth and a sense of connection with people.
2. Collaboration with MUFG
In fiscal 2017, we held a workshop-style discussion
to explore a joint study on the theme of realizing thefuture vision of banking services. About 20 members
from MUFG participated in regular meetings, mainly
from the Digital Transformation Division, who arepromoting digital transformation of the banking business. The scale of the joint study became very large
as various studies were initiated and discussed ineight meetings in total.
In these studies, we exchanged views on introducing NTT’s technology and the possibility of utilizing
it with two major points of focus, namely, a newvideo experience, and a communication engine andtools.
To illustrate the first point, a new video experience,
we held a joint exhibition using a television-size
viewing box on how to use Kirari! with customers
when they visit branches. This conceptual exhibition
provided exhibition visitors with a hands-on experience in which a virtual agent (a character mascot
representing MUFG) was projected in a pseudothree-dimensional manner by Kirari! technology to
NTT Technical Review

Vol. 17 No. 2 Feb. 2019deal with customers. This exhibition received a favorable reception at the Singapore FinTech Festival 2017
and CEATEC JAPAN 2017.
For the second point, a communication engine andtools, we introduced our research accomplishmentsin speech technology cultivated by NTT over many
years, for example, speech recognition, speech synthesis, and intelligent microphones, and discussed
how to utilize these technologies in the banking business. For example, we hypothesized that linking theintelligent microphone and speech recognition withspeech-to-text technology would enable voice data
recorded while dealing with visiting customers to beconverted to text, which could then be accumulated
and analyzed. Sharing good responses to customer
inquiries as examples among branches would make itpossible to improve customer satisfaction and work
efficiency.
We also introduced our annotation technology,
which displays operational explanations and messages regarding issues needing attention on the system screen without affecting the existing system.
Accordingly, we were able to obtain opinions on specific usage scenarios stating that the technology canbe utilized as a system to support operations byelderly people and foreign customers as banking procedures continue to advance toward being paperless
and are handled by customers themselves.
From this point on, we will continue to strengthen
our relationship through a series of joint-study sessions in order to promote digital transformation in thefinancial industry (Fig. 1).

33

Feature Articles

Transparent organic electroluminescence display
Operator at remote location“Hello”
Object extraction
by Kirari! (PC)PC
Teleconference terminal

Network

Teleconference terminal

PC: personal computer

Fig. 2. Prototype of customer reception system.

3. Utilization of latest technologies
NTT and NTT Communications are making fulluse of the knowledge gained through these studies tocommercialize products and services.
3.1 Speech recognition
The joint study enabled us to recognize the possibility of utilizing the speech-recognition and Japanese-language-analysis technology of the NTT laboratories in MUFG operations. As a result, the application of this technology is being considered for callcenters as well as in other scenarios such as at storefronts, for outside visits, and for recording the minutes of meetings at banks to improve work efficiency
by combining technologies that can visualize the customer’s voice by speech recognition and can carry out
thorough searches by using robust media search technology. As the first step of one study, MUFG’s outbound call center is utilizing ForeSight Voice Mining
to evaluate the possibility of applying it for purposessuch as emotion analysis and summarization.
3.2 COTOHA TranslatorTM
Automatic-translation technology based on COTOHA TranslatorTM is proving advantageous with theglobalization of the banking industry. It is increasing
the number of opportunities to translate materials andserve customers in multiple languages in scenarios
such as dealing with inbound visitors to banks.
COTOHA Translator achieves high translation accuracy without changing the slide layout of Microsoft
PowerPoint documents; consequently, it is gaining
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

popularity as it leads to significant improvement inwork efficiency, and we are presently evaluating it at
MUFG. To diversely utilize these corevo AI technologies of language recognition and analysis processingin the financial business, we will promote joint studies aimed at providing new banking services by leveraging NTT’s comprehensive technical capabilities
such as the fusion of corevo and Kirari!.
3.3 Kirari!
At the NTT Communications Forum 2018 held in
October 2018, a terminal capable of displaying glassless three-dimensional images was exhibited. Thisterminal uses Kirari! technology and an aerial-imaging plate provided by NTT Communications in cooperation with Dai Nippon Printing Co., Ltd. We areworking to offer the terminal at a reasonable cost andin a size used at financial institutions and otherbranches, and are planning to commercialize the terminal in the future.
In addition, the NTT laboratories exhibited a prototype of a customer reception system at the NTT R&D
Forum 2018 Autumn (Fig. 2). This system connects
the object extraction technology used in Kirari! witha general-purpose teleconference system and uses atransparent organic electroluminescence display. It
can provide a realistic sensation by Kirari! in a general broadband environment such as that provided by
a FLET’S Internet access service. We want to makethe system available to a wide range of industries—
starting with the banking industry—that must deal
with customers in remote locations.

34

Feature Articles

4. Future directions
The NTT laboratories are researching and developing cutting-edge technologies such as blockchain andquantum computers that are expected to be used in
the fintech field. We will continue to strive to derive
the best practices through collaboration with MUFG
and other financial institutions.

Cooperation with NTT Group and future expectations
At MUFG, digitization is one of the pillars of the structural reform outlined in our medium-termbusiness plan, and the Digital Transformation Division is playing a role in developing and promoting
wide-ranging initiatives for improving top-line results and streamlining processes. In regard to thoseinitiatives, NTT Communications has introduced new technologies of the NTT Group in cooperation
with NTT researchers through workshops, tours of the Musashino and Yokosuka R&D Centers, NTT
Communications Forums, etc., and we have received many ideas on new things we can do.
We are currently working with NTT Communications on converting voice data into text at outboundcall centers. I expect to see more proposals that will lead to a reduction in workloads by making full useof the NTT Group’s advanced speech and transmission technologies.
Fumitoshi Imafuji
Director, Digital Transformation Division,
Mitsubishi UFJ Financial Group, Inc.

NTT Technical Review

Vol. 17 No. 2 Feb. 201935

Feature Articles

Takashi Tominaga

Senior Manager, Strategic Business Creation
Team, Research and Development Planning
Department, NTT.
He received a B.S. in physics from Ehime
University in 1995. He joined NTT in 1995 andworked in the corporate sales and service creation department of NTT WEST. Since 2017, hehas been responsible for research and service
creation in the area of financial technology.

From left: Yoshihisa Koike, Manager, Marketing, Third Sales Division, NTT Communications Corporation; Fumi Enomoto, Senior Account Executive, Financial
Industry Sales Group, Third Sales Division, NTT Communications Corporation;
Rika Miyahara, Senior Manager, Financial Industry Sales Group, Third Sales Division, NTT Communications Corporation; Seiichiro Kitamaki, Account Supervisor,
Financial Industry Sales Group, Third Sales Division, NTT Communications Corporation

Takashi Miyatake

Senior Researcher, Natural Communication
Project, NTT Service Evolution Laboratories.
He joined NTT in 1995. He moved to NTT
Communications in 1998 and worked on system
integration of corporate systems. He rejoined
NTT in 2014 and engaged in research and business development of digital watermarking, information and communications technology for thehome, and big data. He has been researching anddeveloping Kirari! immersive telepresence technology since 2015.

NTT Technical Review

Vol. 17 No. 2 Feb. 201936

Feature Articles: Value Creation by Leveraging Stateof-the-art Research Results

Initiatives in the Primary Industries(Agriculture, Forestry, and Fisheries)
Utilizing Advanced Technology of the
NTT Group
Yoshikazu Kusumi
Abstract

Currently in Japan, primary industries such as agriculture are facing problems including a shrinking
working population, an aging working population, and a decreasing area of agricultural land. Agritech—which utilizes the Internet of Things, big-data analysis, artificial intelligence (AI), and robot technology
in the agricultural industry—is attracting attention as a trump card to solve these problems. In this article,
concrete initiatives involving agritech, focusing on utilization of the NTT Group’s AI-related technology
called corevo®, are introduced, and future directions are discussed.
Keywords: agritech, AI, animal husbandry, rice cultivation

1. Increased focus on Agritech
Japan’s agriculture industry is facing problems
such as a shortage of labor due to a decrease in thenumber of agricultural workers and low productivity.
In recent years, attention has been paid to agritech*1—which utilizes the Internet of Things (IoT),
big-data analysis, artificial intelligence (AI), androbot technology in the agricultural industry—as akey to solving these problems.
In particular, efforts utilizing AI have started bothin Japan and abroad as support for and substitution of
the eyes and brains of agricultural professionals(exemplary farmers), whose numbers are steadilydecreasing. The NTT Group is also engaged inresearch and development and service creation ofagritech unique to telecommunications carriers by
utilizing its AI-related technology called corevo®. We
are exploring its use for verification and demonstration purposes in the areas of rice cultivation, dry-field
farming, and livestock farming as well as promotingsocial implementation of our agritech services.

2. Technology applications in the animalhusbandry field
Applications of NTT technology in farming havebeen ongoing for some time, and one important area
is animal husbandry. Two examples of such applications are described in this section.
2.1 Technology use in breeding cows
In farm management, it is very important to grasp
livestock information on individual animals andherds such as the state of health and signs of estrus
and labor required for efficient breeding in a timely
manner. NTT TechnoCross is providing real-timevisualized information via a data-analysis platform
called IoT Data Analysis Suite—which identifies
seven key behaviors (i.e., feeding, drinking, ruminating, moving, standing up, lying down, and standing
still) from sensor information recorded by tags
attached to cows—as part of a service called
U-motion® provided by desamis Co., Ltd., a leader in*1 Agritech: The combination of agriculture and technology.

NTT Technical Review

Vol. 17 No. 2 Feb. 201937

Feature Articles

Estrus index

Sign of estrus

Amount of activity

Standing
and lying down
Ruminating

Cow activity timeline

Feeding
and moving
Source: Created using materials of desamis

Fig. 1. Screen shot of U-motion®.

the animal husbandry field.
We also provide a service to notify the farmer of a
cow’s condition in terms of signs of estrus, disease(poor condition), and difficulty in standing up at
appropriate timings by analyzing the accumulated
data (Fig. 1). In particular, the effect of an alert function that promptly detects cows with standing difficulty (which risk sudden death if they cannot standup) is significant in the beef cattle industry, in which
individual animals are traded at 1 to 1.5 million yen(8000 to 13,000 US dollars). As the introduction of
U-motion advances, more cases of cattle with standing difficulty are being detected and accumulated,
and analysis accuracy is improving accordingly.
Moreover, if a dairy cow becomes sick, milk yield
will be affected. Utilizing U-motion makes it possible
to promptly detect and deal with diseases, and it isthus expected to have various beneficial effects suchas minimizing decreases in milk yield.
Pastures of cow-feed grass also have a major
impact on the amount and quality of milk. For example, when fog forms at the time the grass is being cut,
the grass moistens, and the quality of feed for the
livestock deteriorates; occasionally, all of the feedmust be discarded. Also, if the feed has poor quality
as a result of that moisture, the cows lose their appetite, and milk output falls. However, it is difficult tograsp the generation of fog because it is difficult todistinguish it from clouds, so expectations for new
technology in the livestock field have increased.
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Accordingly, NTT Communication Science Laboratories (hereafter, CS Labs), HALEX Corporation of
the NTT DATA Group, the Japan Meteorological
Agency, the Hamatonbetsu Evergreen TMR (total
mixed rations) Center in Hokkaido (a cow-feedingservice center), and other bodies have collaborated in
integrating weather-related big data collected fromthe meteorological satellite Himawari using techniques such as spatio-temporal data analysis by CS
Labs (part of the corevo AI technologies) and theexpertise of HALEX as a private weather company.
The aim of this integration is to predict the risk of fog
development and support decision making at grass
harvest time before fog forms (Fig. 2).
2.2 Application to breeding pigs
There are various challenges involved in the breeding of pigs. One crucial issue is weight control before
shipping because pig prices are determined by theweight of a pig at the time of shipment. Accurately
grasping the pig’s weight is also important in managing a pig’s health. The required feed changes as piglets grow into pigs, and if the appropriate feed is not
given at the appropriate timing, the pig’s health andthe quality of its meat will be affected. Consequently,
it is necessary to give feed that matches the bodyweight of the pig.
The method of weighing pigs at the time of shipment is to use scales exclusively designed for pigs toweigh pigs individually or to conduct a collective
38

Feature Articles

It is currently difficult to grasp andpredict fog generation.

Numerical forecasting

We are working on predicting fog
generation by utilizing AI etc.

Fog-generation prediction• Calculates atmospheric andoceanic conditions based on
observation data
• Corrects calculated values
and reduces error
Supercomputer of Japan
Meteorological Agency

Open data etc. of Japan
Meteorological Agency

Japanese
Weather
Meteorological BusinessAgencyConsortium

Real-time weather data

Spatio-temporal data
analysis technology

Monitoring camera
Data utilization
know-how of
weather companies

Geostationary
meteorologicalsatellite Himawari

HALEX

Application scenario(agriculture)
Prediction of fog generation isuseful for controlling feed-crop
growth and deciding the time
to reap harvests.

Fig. 2. Conceptual diagram of predicting risk of fog generation.

measurement. However, both methods are costly andtroublesome and place a burden on pig farmers. Until
now, the timing of shipping has been determinedaccording to the age of the pig (in months) and theskilled eye (called mekan in Japanese) of an expert, so
more experience of skilled people was needed.
In collaboration with ITOCHU Feed Mills Co.,
Ltd., NTT TechnoCross is developing Digital
Mekan—which applies measurement logic incorporating corevo AI image-recognition technology toaccurately estimate the weight of pigs (Fig. 3).
Although Digital Mekan was originally developed as
an application for smartphones, it is currently being
developed as a packaged solution including dedicatedhardware. It is almost complete, and a dedicated unit
is being developed and set for early release.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

3. Applications to rice cultivation
The agriculture, forestry, and fisheries industries
operate in step with nature, and consequently, they
can suffer from damage peculiar to the natural world.
One such example is damage from pests. On a global
scale, it is said that 20–30% of agricultural production capacity is lost because of damage caused bypests and weeds. A loss on that scale amounts to aquantity of food for hundreds of millions of people,
equivalent to the hunger population of the world.
Meanwhile, the burden on farmers to come up withcountermeasures such as dealing with climatechange, diversifying cultivated crops, and guarding
against invading pests from overseas as a result ofinternationalization of agricultural distribution, aregetting ever heavier.
As a member of a project of the Ministry of Agriculture, Forestry and Fisheries of Japan, NTT DATA
39

Feature Articles

Source: NTT TechnoCross document(a) Measured image

it is necessary to know precisely the timing of thestart of panicle differentiation and to provide additional fertilizer*2 at the appropriate time. However,that timing is presently determined based on the longexperience and intuition of exemplary farmers or bya more scientific approach such as microscopic
examination. These approaches, however, are accompanied by the twin problems of a shortage of successors and the large amount of labor required.
In collaboration with Ibaraki Prefectural Agricultural Experiment Station, DATA CCS has taken theinitiative in constructing a mechanism to determine
the start of differentiation of panicles by employing
deep learning (a kind of nondestructive testing). In
particular, each growth stage is classified using
images taken by a fixed camera, and the shape of therice at the time of capturing the image is determined.
We are currently expanding this mechanism to coverother crops and deploying it in a wider region. We
have already applied for international patents on thissystem.
4. Future development

Source: NTT TechnoCross document(b) Displayed image

Fig. 3. Digital Mekan for estimating pig’s weight.

CCS (hereafter, DATA CCS) is working with Nihon
Nohyaku Co., Ltd. to develop a diagnostic system toidentify pests and weeds in rice fields. By combining
image analysis utilizing AI technology of DATA
CCS, the knowledge and expertise of Nihon Nohyaku
on pesticide manufacturing, and the large amount ofpest images possessed by Nihon Nohyaku, we areaiming to construct a system to provide real-time
feedback of information such as potential pests, theirrespective countermeasures, and recommended pesticides to farm workers. This system, for example, will
enable a farmer or an agricultural advisor to match aphoto of a pest taken with a smartphone with images
stored in a pest database (Fig. 4).
In addition, as a new initiative, we are developing amechanism for discriminating the growth stage of
paddy rice by using deep learning with images,
focusing on changes in the shape of paddy rice. The
rice-growing cycle is roughly divided into five stages:
tilling, panicle differentiation, meiosis, heading, andripening. To raise the yield, taste, and quality of rice,
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Agritech is moving from the verification and demonstration phase to the social implementation phase.
Under these circumstances, companies in the NTT
Group have been individually promoting productdevelopment and deployment. From now onwards,
however, we will accelerate the construction of amechanism for linking each product and accumulated
data in such a manner that will strengthen our competitiveness.
For example, with regard to rice cultivation, by
linking together the above-mentioned mechanisms
for pest and weed disease diagnosis, growth diagnosis, sensing technology and weather and map information of the NTT Group, and future forecasting
technology under development in our laboratories,
we aim to establish a total diagnosis and prediction
service for rice cultivation (Fig. 5). Moreover, bycoordinating the rice production utilizing ICT (information and communication technology) with the
NTT Group’s digital food value chain linking agriculture and food [1], we are establishing a mechanism
for supporting people involved in agriculture in termsof sales as well as production. By doing so, we aim tocreate agriculture that is profitable from the aspectsof both saving labor and increasing profit.
*2 Additional fertilizer: Fertilizer applied after seeding or transplantation.

40

Feature Articles

Agricultural producers/
technical advisors

Advanced image
diagnosis
NTT Group
Request for diagnosis
Pesticide
manufacturer+
Send image information(pests) and information
on surroundings.

Time/date

Size

Location

Crop

Temperature

Humidity• Potential pest
• Recommended
pesticide• Countermeasure

Pesticide purchase
via website

Database ofpest pictorial
books (classification
information + images)

Learning
data

Narrow down and provide
candidate pest informationfrom past learning data.
Results of diagnosis

Advice on accurate diagnosis
and pesticide information

Fig. 4. Illustration of pest diagnosis system.

Step 2: Enclose users.
Growth prediction

Step 1: Attract users.
Matching supply and demand

Growthdiagnosis
Pest diagnosis
Agricultural
crops DWH
(pest and growth
diagnosis)

Pest prediction

Handling extraordinary weather(insurance etc.)

Detailed weather
and maps

Fertilizer andpesticidespraying

Creating register of agricultural landStart
Sensing technology
Weather forecast
DWH: data warehouse

Step 3:
Toward the agriculturalbusiness of the future(such as yield guarantee)

Trading company etc.

Fig. 5. Illustration of total support service for rice cultivation.

Furthermore, we are thinking beyond simply
exporting rice produced in Japan and are planning to
expand into Asian countries where rice cultivation isas prosperous as in Japan. That is, we aim to roll out
NTT Technical Review

Vol. 17 No. 2 Feb. 2019the above-mentioned rice production support mechanisms created in Japan under the concept of Made by
Japan.
From now onwards, we will continue to contribute
41

Feature Articles

to the development of primary industries with a viewto globalization with the aim of making the NTT
Group a value partner that will continue to be selected by numerous companies.
Reference[1]

Y. Kusumi, “B2B2X Collaboration in the Primary Industry Sector:

Agriculture, Forestry, and Fisheries,” NTT Technical Review, Vol. 17,
No. 1, pp. 14–19, 2019.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr201901fa4.html

Trademark notes

All brand names, product names, and company/organization names thatappear in this article are trademarks or registered trademarks of theirrespective owners.

Yoshikazu Kusumi

Senior Manager, Strategic Business Creation
Team, Research and Development Planning
Department, NTT.
He received a B.S. and M.S. in nuclear engineering from Osaka University in 1993 and 1995.
He joined NTT in 1995 and was assigned to thelong distance communication business division.
He then worked at the network engineering center, where he designed a network system forpersonal handyphone systems. He also worked at
NTT Communications in the carrier sales division, where he was in charge of wholesale business for foreign companies and later, for planning global strategies. He is at NTT again, where
he is responsible for ICT business creation for the
agriculture domain.

NTT Technical Review

Vol. 17 No. 2 Feb. 201942

Feature Articles: Value Creation by Leveraging Stateof-the-art Research Results

Improving Amenities at Narita
International Airport—Utilizing
High-precision Indoor Maps andGeomagnetic Positioning to Facilitate
the Movement of Visitors
Tomohiko Murakami, Hitoshi Seshimo, Wataru Kondo,
Tomo Inaba, Mitsuaki Takaishi, Shinji Yarinome,
and Shinnosuke Nakamura
Abstract

The number of international visitors to Japan has been soaring. At Narita International Airport, thegateway to Japan, we aim to create airport services that allow everyone to spend time comfortably via
the concept of a smart airport. In September 2018, a service called NariNAVI—Japan’s first high-precision indoor navigation application introduced at an airport—was launched. This application utilizes
geomagnetic positioning as indoor-positioning technology. To display the map, we adopted a 2.5D (twoand-a-half-dimensional) map, which stereoscopically expresses spaces inside complex airport terminals
spanning multiple floors. Providing such a map that is intuitive and user-friendly supports customers’smooth movement inside the airport.
Keywords: NariNAVI, geomagnetic positioning, 2.5D map

1. Current status and issues concerning
Narita International Airport
In 2018, Narita International Airport (Narita Airport, hereafter) celebrated its 40th anniversary since
starting operations. It is one of Japan’s busiest airports, so it is necessary to ensure that it is capable ofhandling the influx of visitors that continues toincrease each year.
1.1 Increasing number of international visitors
In anticipation of the international sports event
being held in Japan in 2020, the country has madegood progress in regard to national tourism policy.
The number of inbound visitors has increased fromNTT Technical Review

Vol. 17 No. 2 Feb. 20198.36 million in 2012 to 28.69 million in 2017, morethan a threefold increase, and the government has set
the target of 40 million visitors in 2020.
Narita International Airport Co., Ltd. (NAA) has
been promoting a three-year NAA Group mediumterm management plan—Innovative Narita 2018—
since 2016. This plan aims to retain and reinforce
Narita’s position as a top-class international hub airport in Asia while striving to become an airport providing the world’s highest standard of quality andservice for customers. By creating the world’s highest-level smart airport by utilizing state-of-the-art
information and communication technology, NAA is
striving to impress customers by providing them withexciting and unexpected experiences while setting the43

Feature Articles

smart airport initiative as its aim to improve customer
convenience and the overall comfort of Narita Airport.
1.2 U tility of high-precision indoor electronicmap
The space in an airport is vast. As well as acting asa transportation node for buses and railways, it functions as a complex commercial facility. People comefrom all over the world to visit Japan, and our experience tells us that the needs of those passengers arechanging. For example, access to the city center,
Wi-Fi service, and souvenir shops are just some of theneeds to be met. We cannot respond to these diversified needs with only a simple unified map, so wetherefore decided to create a high-precision indoor
electronic map.
Creating electronic mapping of existing buildings
takes time and effort because the points where mapsof indoor areas and outdoor areas connect differ
depending on the map data sources, and there is no
unified standard for map data. NAA prepared an electronic map in compliance with the guidelines createdthrough the High Accuracy Positioning Society Project initiated by the Ministry of Land, Infrastructure,
Transport and Tourism of Japan.
In October 2017, we developed a form of digital
signage called infotouch, which utilizes a high-precision map, and digitizing that map has made it possible to reflect changing terminal information in atimely manner. In addition, we introduced a mechanism for continuously displaying pictograms on themap of the facilities frequently used by customers(toilets, smoking areas, etc.) so that customers do notneed to search for such facilities. Furthermore, theuniversal design approach was incorporated fromscratch in various aspects of the designs of colors,
buttons, and other details. We are installing four
infotouch units in Terminal 1 and plan to expand theinstallation to cover all airport buildings by the end of
fiscal 2019.
As a new initiative, we simultaneously developed
an application programming interface (API) thatmakes it possible to use high-precision electronic
maps in other operations and by airline companies.
We surmised that other businesses located at the airport would use the API to provide information displayed by infotouch on mobile devices and thereby
improve customer convenience. By using location
information in this manner, we can offer new ways tospend time at the airport.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

1.3 Creation of NariNAVIDevelopment of the navigation application called
NariNAVI (Fig. 1) started with the above-mentioned
idea in mind. There are two important factors regarding information displayed via the application: theaccuracy of the user’s position (self-location) and theease of setting the destination. Various technologies
are available for indoor positioning. We selected
iBeacon since it is being adopted internationally as amainstream technology for airport infrastructure. We
installed about 1500 iBeacons at Narita Airport, andwe combined the use of the iBeacons with geomagnetic positioning to determine location. Althoughinstalling the iBeacons in the proper position andheight was difficult due to constraints such as passenger flow lines, we overcame that difficulty byreconsidering the typical geomagnetism survey
method. It seems natural, but specifying a self-location indoors in places that GPS (Global Positioning
System) cannot reach is actually quite groundbreaking.
We implemented a predictive conversion functionin the destination setting of NariNAVI. Althoughmost search logics provided by facilities will ask fora search by formal name, we created a dictionary that
also supports several variations so that users cansearch for a target facility without entering its official
name. This feature makes it possible to search forflight information and shop facilities more easily.
Another feature of NariNAVI is that it can be usedwith web browsers other than Android and iOS applications. It is displayable by web browsers, so it iseasy to link with various web services via homepages. Moreover, by extending NariNAVI to robotics andchatbots (which are progressing in parallel with services based on location information), NAA wants tooffer new services to everyone who uses Narita Airport.
2. Technology of NariNAVI
Two NTT Group technologies were adopted in
NariNAVI to accurately determine indoor locationand intuitively represent maps in the complex layoutof a multilevel airport. One is high-precision indoorlocation technology using geomagnetism, and theother is two-and-a-half-dimensional (2.5D)-map
platform technology. Geomagnetic positioning is afunction of a cloud service called high-precision
location information provided by NTT DATA, and it
utilizes technology jointly developed by NTT DATA
and GiPStech. The 2.5D-map platform technology
44Feature Articles

Terminal 2

Terminal 1, 3F (third floor)
Current displayed location

Current-location
button2.5D map display
A mode to stereoscopically display multiple
hierarchies. If the destination in the building
is on another level, a 2.5D illustration
enables users to intuitively understand the
direction to follow.

Display icon selection(a) Top screen(b) 2.5D map

2.5D: two and a half dimensional

Fig. 1. NariNAVI.

was developed by NTT Service Evolution Laboratories and is used to express and transmit maps. NTT
DATA is developing the NariNAVI application, andthey have incorporated the above-mentioned technologies in the application.
2.1 H
 igh-precision indoor location-information
service
The high-precision indoor location-information
service provided by NTT DATA is a cloud service
providing map distribution, indoor positioning, androute-search functions, which are basic functions forcreating navigation services in indoor spaces. By
using this service, customers who wish to create
indoor guidance services can focus on developing
higher-level applications.
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

The mechanism of indoor positioning provided bythis service involves calculating the optimal balance
between radiowaves emitted by various kinds ofequipment such as Wi-Fi and BLE (Bluetooth Low
Energy) routers and other beacons installed in customer facilities and the geomagnetism in the building, and that calculation makes high-precision positioning possible. The high-precision indoor locationinformation service provided by NTT DATA wasselected since NAA aimed to enable high-precisionindoor positioning by effectively utilizing alreadyinstalled iBeacons.
As its name suggests, geomagnetic positioning is
positioning using the Earth’s magnetism. Indoors,
where geomagnetism is distorted by structures suchas steel frames of buildings, the characteristic
45

Feature Articles• Survey target area.

• Create magnetic map.

(a) Advance preparation• Obtain magnetic information by magnetic
sensor of user’s smartphone.

• Guess transition route frommagnetic fluctuation.

• Identify current location.

(b) Indoor positioning

Fig. 2. Mechanism of geomagnetic positioning.

strength or weakness of magnetism is easily manifested; geomagnetic positioning technology utilizes
that characteristic. The intensity of geomagnetism
was measured by conducting a survey on foot at thesite, and the collected data were used to create a database. Position is determined by matching the database values with the magnetic value measured by themagnetic sensor installed as standard in most smartphones (Fig. 2).
Once the geomagnetic data have been measured,
there is no need to remeasure the data as long as there
are no major structural changes such as large-scaleconstruction, and this feature enables stable positioning on a continuous basis.
At Narita Airport up until now, indoor maps and a
geomagnetic-positioning environment were createdin a series of development phases. In other words, thefoundation of highly accurate indoor position information was laid. By utilizing this foundation, various
services can be developed in the future. In addition toproviding infotouch and NariNAVI, we will continue
to collaborate with NAA to further improve services
at airports by, for example, analyzing the behavioral
history of users from log information, improving thework processes of airport staff, and managing goods
NTT Technical Review

Vol. 17 No. 2 Feb. 2019such as luggage carts and strollers.
Also, from a user’s viewpoint, expanding the area
in which users can be guided indoors would enable
the same service to be received anywhere, leading to
improved user convenience. NTT DATA intends toroll out this service at other airports and public transportation facilities in the future.
2.2 Advanced 2.5D-map platform technology
At NTT Service Evolution Laboratories, we havebeen carrying out research and development aiming
to achieve navigation services that provide safe andconvenient support for movement of a diverse groupof people including those using wheelchairs andstrollers, elderly people, and international visitors toJapan. That research resulted in the development of
2.5D-map platform technology, which enables seamless transmission and display of outdoor and indoor
hierarchical maps (i.e., planar maps with floor andheight information).
Unlike automobiles, which basically move in a planar manner, people move in a stereoscopic manner.
Movements of people include coming and going
indoors and outdoors as well as movements up anddown stairs and elevators. With conventional map
46

Feature Articles

Main
gate

Map
display

Main
gate

Map
display
Routedisplay

Routedisplay

Main
gate

Main
gate

Building
6, 1F

Building
6, 1F

Building
6, 3F

Building
6, 3F

Building
7, 1F

Building
7, 1F

Building
7, B1F

Building
7, B1F

Building 6
Entrance

Meeting
space 2Meeting
space 2

Building 6, 1F
Meeting
space 1

Building 6
Entrance
Building 6, 1F
Meeting
space 1Height:Direction:
Up and down:
Clearance:

Height:Direction:
Up and down:
Clearance:(a) Indoor 2D map(b) Indoor 2.5D map*Map data source: NTT EAST

Fig. 3. Automatic generation of 2.5D map.

services, this situation was handled by switchingoverlapping planar (2D) maps; however, with thatapproach, it is difficult to see one’s present location
and a destination on different floors from a bird’s-eye
view, and the guided route is cut off at each level,
making it difficult to understand the route intuitively.
With these circumstances in mind, we simply added
height information (i.e., the number of levels in eachfloor, ceiling height, etc.) as an extra half dimension(+ 0.5D) to indoor 2D floor maps aligned on a conventional outdoor 2D map to create a way to automatically generate, transmit, and present simple stereoscopic maps, referred to as 2.5D maps (Fig. 3).
Changing the map to a stereoscopic representationfrom 2D to 2.5D, as mentioned previously, makes it
possible to view the positional relationships between
different levels connected by staircases and elevators,
and a guided route can be confirmed at a glance at any
time. As a result, the user can see the route clearlyahead of time and, for example, decide whether to goleft or right before going up the stairs. It is also posNTT Technical Review

Vol. 17 No. 2 Feb. 2019sible to display walls or floors transparently, so theuser can foresee things that cannot be seen in the realworld.
Although it does not express images as elaboratelyor realistically as a 3D map does, it reduces the timeand cost of creating maps by utilizing existing planarmaps and indoor floor maps, and the amount of dataread by the user terminal and its processing load canalso be curtailed. This makes it possible to provide
services on a web-browser basis regardless of whether a personal computer or mobile device is being
used. In fact, the maps displayed on infotouch areused as a common mechanism for map display on
both the application and browser versions of NariNAVI.
At NTT Service Evolution Laboratories, we havebeen working on new research and development
while receiving feedback on demonstration tests conducted with the Ministry of Land, Infrastructure,
Transport and Tourism since 2016, joint demonstrations with private companies, and commercial
47

Feature Articles

introduction of the technologies described in thisreport. For example, creating indoor 2D maps (which
are the source of the 2.5D maps) currently requires a
lot of manual labor, so we are studying whether it ispossible to automatically generate 2.5D maps withexisting data used by building-design systems.
Since economization is considered to be one of the
key objectives when rolling out technology, we hope
to increase the efficiency of map-production work as
much as possible by using technology and to improveindoor maps at low cost and in a short period of time.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Moreover, regarding efforts concerning the ease ofunderstanding the navigation while making maximum use of 2.5D map data, we are also studying
guidance methods that do not rely on visual representations (such as maps) for those who are not good atreading maps.
Trademark notes
All brand names, product names, and company/organization names thatappear in this article are trademarks or registered trademarks of theirrespective owners.

48

Feature Articles

Tomohiko Murakami

Supervisor, IT Planning, IT Development andPlanning Development, Corporate Planning
Division, Narita International Airport Co., Ltd.
He is engaged in the development of mobile
applications (NariNAVI, TABIMORI), indoor
positioning, development of various robots(communication robots, cleaning, security, etc.),
AI chatbot operation (Bebot), and development
of digital signage.

Hitoshi Seshimo

Senior Research Engineer, Supervisor, Proactive Navigation Project, NTT Service Evolution
Laboratories.
He received a B.E. and M.E. in mechanicalengineering from Waseda University, Tokyo, in
1995 and 1997. He joined NTT in 1997. His
research interests include computer aided
instruction, web-based learning, content distribution and navigation systems, and geographicalinformation services.

Wataru Kondo

Manager, Sales and Marketing Group 1, Digital Solution Section, Social Innovation Division,
Social Infrastructure Solution Sector, NTT
DATA Corporation.
He joined NTT DATA in 1995. He manages asales team focusing on indoor location services
and smart mobility.

Mitsuaki Takaishi

Deputy Manager, Technology and Development Group 1, Digital Solution Section, Social
Innovation Division, Social Infrastructure Solution Sector, NTT DATA Corporation.
He joined NTT DATA in 2008. He is involved
in developing services utilizing indoor positioning environments.

Shinji Yarinome

Technology and Development Group 1, Digital
Solution Section, Social Innovation Division,
Social Infrastructure Solution Sector, NTT
DATA Corporation.
He received a B.E. from Keio University,
Tokyo, in 2002. He joined NTT DATA in 2002
and is mainly engaged in developing public systems. He was assigned to the Narita International
Airport project in 2017. He developed a precisionindoor digital map system and an indoor navigation mobile application (NariNAVI) as a teamleader. He acquired PMP (Project Management
Professional) Certification in 2013.

Shinnosuke Nakamura

NTT Data System Service Predecessor, NTT
DATA Corporation.
He joined NTT DATA in 2003. He first worked
as a system engineer and has been engaged inbusiness operations since 2006.

Tomo Inaba

Manager, Technology and Development Group
1, Digital Solution Section, Social Innovation
Division, Social Infrastructure Solution Sector,
NTT DATA Corporation.
He joined NTT DATA in 1997. He is responsible for the management and design of geographical information systems and location systems as a project manager.

NTT Technical Review

Vol. 17 No. 2 Feb. 201949

Feature Articles: Value Creation by Leveraging Stateof-the-art Technology

Creating Value from Deep Learning
Technology and Its Business
Applications
Kunihiro Moriga, Takeharu Eda, Masashi Toyama,
Keita Mikami, Yutaka Hirokawa, Yuji Yamada,
Sanae Muramatsu, Taku Sasaki, Shin’ya Yamaguchi,
and Katsuo Inaya
Abstract

NTT has positioned the utilization of artificial intelligence (AI) as an important business strategy. NTT
Software Innovation Center has been engaged in research and development aimed at creating value fromthe AI technology known as deep learning. In this article, we explain how the deep-learning inference
environment can be optimized for business applications, namely by speeding it up, making processing
lightweight, and saving labor. We also report on verification activities concerning the image-analysis
business opportunities (such as surveillance-camera image analysis) generated by using that technology.
Keywords: deep learning, cloud-based inference, optimization

1. Deep learning: entering thedisillusionment phase
Artificial intelligence (AI) is currently said to be in
its third boom. The Hype Cycle for Emerging Technologies, 2018 [1] published by Gartner Inc. inAugust 2018, predicted that deep learning would be
at a peak of inflated expectation in its second consecutive year. In other words, it would soon enter adisillusionment phase.
Conditions such as PoC (proof of concept) andprecedent cases and best practices were published by
cutting-edge companies; however, the persons anddepartments in charge might be feeling the difficultyof solving problems using deep learning technology
by themselves.
During the peak of inflated expectations phase, theresults and utility that everyone imagined and expected cannot be obtained, and the people who are in
charge become disappointed as they face the real situation; this is the beginning of the disillusionment
NTT Technical Review

Vol. 17 No. 2 Feb. 2019phase. However, that situation can be said to be thebeginning of true business applications. From now
onwards, implementation and peripheral technologies of deep learning will catch up, and deep learning
will gradually be adopted in actual business operations.
The technology developed by NTT Software Innovation Center that can analyze images of people inreal time is an example of such technology that is
approaching the stage of business application.
2. Analysis of images captured by multiple
surveillance cameras at high speed in real time
Since deep learning became a topic of research inabout 2011, it has succeeded in giving something
akin to a person’s eyes and ears to computers. Moreover, as of 2018, it is no exaggeration to say that at thepurely technical level, it has already passed beyond
the human eyes and ears ability. Real-time person
tracking developed by NTT Software Innovation50

Feature Articles

When people are detected in the video image,they are enclosed in a blue frame. The line
extending from the bottom of the blue frame isthe estimated trajectory.

As shown in the figure above, if a planview of the room is prepared in advance,
it is possible to map the trajectory onto
the plan view.

(a) People detection(c) Trajectory recognition

After people are detected in the video image by the people
detection function, if a photo (as shown on the right side of
the figure) of a person being looked for is specified, when thespecified person is detected in the image, the blue frame
switches to a red frame, which indicates that the specified
person has been found. Moreover, in this video, it is possible
to identify the same person even with a backward-looking
video by performing the whole-body collation function
described in section 2.1.
(b) Person re-identification

Fig. 1. Functions of real-time person tracking.

Center is a form of deep learning packaged as imageanalysis technology [2]. It analyzes images captured
by a large number of surveillance cameras installed ina facility in real time and instantly detects and tracks
target persons (suspicious persons, prominent persons, people needing care, lost children, etc.) in thoseimages. Real-time person tracking is enabled by combining the following seven functions (Fig. 1):(1)	People detection: Only people are extracted
from an image (Fig. 1(a)).
(2)	Attribute recognition: The gender and age
group of a person are estimated.
(3)	Detailed attribute recognition: Attributes associated with specific body parts are recognized.
For example, a person is searched for on thebasis of detailed attributes concerning theirappearance, colors of clothing, and presenceof personal items such as a bag (e.g., having
long hair and wearing a white shirt, jeans, and
NTT Technical Review

Vol. 17 No. 2 Feb. 2019sunglasses).
(4)	Person re-identification: Whole-body collation is applied to determine whether thedetected person is the target person (Fig. 1(b)).
(5)	Trajectory recognition: The trajectory of aperson walking is estimated from video images (Fig. 1(c)).
(6)	Multiple camera compatibility: Functions (1)to (5) are supported even if the target person
crosses the views of multiple cameras.
(7)	Real-time analysis: The results of (1) to (6)can be analyzed in real time.
2.1 Achievement of whole-body collation ahead of
our competitors
We achieved function (4), person re-identification,by employing whole-body collation ahead of ourcompetitors, which makes it possible to extract people
even if the person is facing backwards. Automatically51

Feature Articles

extracting features from a large number of pairs ofimages of people by using deep learning technology
made it possible to match the images with higher
precision than rule-based judgment using human-set
characteristics (body type, clothing color, hairstyle,etc.).
The person-search service provided by NTT Communications called Takumi Eyes—which incorporates part of our real-time person-tracking technology—was awarded the 20th Automatic Recognition
System Grand Prize by the Japan Automated Identification System Association in 2018 [3]. Winning theaward indicates that this technology is highly evaluated by the market.
2.2 R esults of collaboration between PanasonicGroup and NTT Group
Whole-body collation is not perfect by itself. A
whole-body check is difficult to perform if a person’s
appearance changes such as when they take off outerwear (coats etc.) that they were wearing. We deviseda solution to this problem in collaboration with ourpartner.
A business alliance agreement with Panasonic Corporation in 2015 [4] triggered efforts to greatly
improve recognition accuracy by combining ourwhole-body-collation technology with Panasonic’s
face-recognition technology [5]. Created as a resultof the combined technologies was real-time person
tracking, which can match people with high accuracy
from camera images shot under various angles andconditions. Until now, we were not aware of any otherservices that combined full-body collation technology and face authentication using deep learning in thismanner. At the present time, only the person re-identification is achieved by whole-body collation plus
face authentication; however, the technology can be
combined with additional detection functions, suchas detecting the gait of a person, in response to theneeds of our customers.
3. Video monitoring market forecasted to
be 160 billion yen by 2030
This section focuses on the business potential of theabove-described real-time person tracking using deeplearning technology.
The image-analysis business is a promising sector
with the highest growth rate in the AI market. It is
expected to grow more than one-hundred-fold, namely, from 1.3 billion yen in 2015 to 160 billion yen in
fiscal year 2030 [6]. It is expected that the market for
NTT Technical Review

Vol. 17 No. 2 Feb. 2019analyzing images shot by surveillance cameras will
increase.
A use case of an actual implementation is describedin the following subsection.
3.1 U
 tilization of surveillance cameras at convenience stores: person re-identification
An example that is easy to imagine as a use caseinvolving surveillance cameras is the use of surveillance-camera images taken in convenience stores.
Having surveillance cameras in present-day convenience stores enables people to confirm what actually
happened from past images after an incident or accident has occurred. A convenient function in such acase is person re-identification.
In the case of a crime occurring at a convenience
store equipped with surveillance cameras, it is possible to quickly find when the perpetrator entered thestore by specifying the image of the person from thevideo at the time of the crime and then searching fromother past video images. In addition, it is possible toquickly find out if the crime was planned or impulsive
by retrieving the perpetrator’s past images in order todetermine the history of their store visits—if they had
been to the store to check it out in the past. Moreover,person re-identification is even more effective in thecase of large facilities such as apartment blocks or
shopping malls fitted with multiple surveillance cameras.
Scenes in television crime dramas in which a police
detective spends a long time checking surveillancecamera videos will surely be a thing of the past once
this technology is put into service. Furthermore, utilizing this technology will eliminate oversights due tohuman error.
3.2 U tilization of surveillance cameras in commercial facilities: finding lost children by
combining attribute recognition, color search,and person re-identification
Finding lost children is an expected use case forlarger-scale commercial facilities. When shopping at
department stores or shopping malls, we sometimes
hear announcements about lost children. Announcements such as, “A mother is looking for her five-yearold daughter (name), who is wearing a pink dress.”are commonly heard on busy weekends at commercial facilities. Such announcements, however, may nolonger be necessary once this technology is put into
service. The child’s age is specified by a techniquecalled attribute recognition, and the color of thechild’s clothes is specified and searched for via a52

Feature Articles

1

A child becomes lost.

2

Searching for an image of the lost child“A five-year-old girlwearing a pink dress.”

3

Monitoring by live cameras

4

The child is safely returned to parent.

The childis found.

Fig. 2. Use of surveillance cameras in commercial facilities to look for a lost child.

color search. From the images captured by nearly 100
surveillance cameras, the most-recent captured imageof the child can be displayed, and the lost child can be
pinpointed from that image (Fig. 2). Such a system islikely to be essential for future large-scale themeparks and other such facilities that do not implement
measures for announcing lost children.
3.3 S
 earch for wandering citizens for municipalities
The example mentioned in the previous section is ause case concerning a lost child in a commercial facility, but it can also be applied to finding elderly people
who wander away from home. With one photo of theelderly person who has wandered away, it is possible
to promptly find that person from the images captured by multiple surveillance cameras operated bymunicipalities. The person re-identification function
can find people in a much shorter time compared tomanually checking surveillance-camera images withthe human eye. Using this technology as a humanassistant makes it possible to promptly find the wandering elderly person and thereby reduce the probability of that person being involved in an accident.
The news footage of people in local municipalities,
including hundreds of local police and fire fighters,
simultaneously searching for elderly people who
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

have wandered away may not be very common after
this technology is put into service.
3.4 U
 se in marketing: combining person re-identification, attribute recognition, and trajectory recognition
This real-time person-tracking technology can also
be used for purposes other than crime prevention,
namely, marketing. It may also be useful for analyzing and outputting the number of visitors and their
attributes in a time slot from surveillance camera
images as well as analyzing the flow of customers. It
will be possible to analyze data and determine how toarrange shelves and goods in a limited store space.
Other applications are also possible. For example,
although a logic program is required for detecting
objects or people, by preparing training data andlearning from it, it will be possible to extract items
that customers had taken from a shelf but returned tothe shelf (that is, products that a customer seemedinterested in but did not purchase). With the introduction of such technology, it will be possible to obtain
information that cannot be extracted from cash-register POS (point of sales) data.

53

Feature Articles

4. Core technology that enables
real-time processing
One particular core technology is important forestablishing real-time processing. That technology isa technique for optimizing the deep-learning inference environment. This optimization technology
makes it possible, for example, to detect and classify
objects in video images at high speed and process
them in real time. By combining the world’s latest
technologies listed below according to the type ofdeep learning processing (i.e., detect and classify)that is executed, we have increased the processing
speed by more than 10 times.
•	Image-analysis algorithm for detecting andmatching people with high accuracy•	Reduction of parameter size•	Implementation technology for optimizing inference processing of deep learning
When the person-tracking service was introduced
for the first time in the world as Takumi Eyes, themain service was searching for past images by using
images captured by surveillance cameras. Since then,real-time processing has become possible by researching and combining optimization techniques for deeplearning inference environments over time.
5. Future development
After developing the video-analysis technologythat enables real-time processing, we plan to focus
our research on distributed processing of video-analysis technology. Specifically, we are planning toconduct research on distributed processing thatNTT Technical Review

Vol. 17 No. 2 Feb. 2019enables systems (including central offices and datacenters of telecommunication companies as well as
cloud services) to be constructed and the functions
required at each location (edge) to be provided. We
call this processing a two-layer edge model. We will
work to make the operation of edge devices more
efficient, lower in cost, and with the maximum savings of power and memory.
From now onwards, the NTT Group will continue
to work towards applying its AI technology called
corevo® and implementing it in society in cooperation with various partners in order to improve thelives and businesses of many customers.
References
[1]

[2]

[3]

[4]
[5]
[6]

Press release issued by Gartner, “Gartner Identifies Five Emerging
Technology Trends That Will Blur the Lines Between Human andMachine,” Aug. 20, 2018.
https://www.gartner.com/en/newsroom/press-releases/2018-08-20gartner-identifies-five-emerging-technology-trends-that-will-blurthe-lines-between-human-and-machine
T. Eda, S. Muramatsu, K. Mikami, S. Xu, “A Practical Person Monitoring System for City Security,” 15th IEEE International Conference
on Advanced Video and Signal Based Surveillance (AVSS 2018),
Auckland, New Zealand, Nov. 2018.
Press release issued by NTT Communications on Sept. 12, 2018 (inJapanese).
https://www.ntt.com/about-us/press-releases/news/article/2018/0912.
html
Press release issued by NTT and Panasonic, “NTT and Panasonic Tie
Up for Innovative Visual Communication,” June 17, 2015.
http://www.ntt.co.jp/news2015/1506e/150617a.html
Press release issued by NTT and Panasonic on Oct. 3, 2018 (in Japanese).
http://www.ntt.co.jp/news2018/1810/181003a.html
Press release issued by Fuji Chimera Research Institute on Nov. 28,
2016 (in Japanese).
http://www.group.fuji-keizai.co.jp/press/pdf/161128_16095.pdf

54

Feature Articles

Kunihiro Moriga

Yuji Yamada

Senior Research Engineer, Supervisor, Distributed Data Processing Platform SE Project, NTT
Software Innovation Center.
He received a B.S. and M.S. from Yokohama
National University in 1991 and 1993. He joined
NTT in 1993. His current research interests
include deep learning and AI base technology.

Engineer, Distributed Data Processing Platform SE Project, NTT Software Innovation
Center.
He received a B.E. and M.E. in computer science from The University of Electro-Communications, Tokyo, in 2009 and 2011. He joined NTT
in 2011. His research interests include data science and software engineering.

Takeharu Eda

Sanae Muramatsu

Senior Research Engineer, Distributed Data
Processing Platform SE Project, NTT Software
Innovation Center.
He received a B.S. in mathematics from Kyoto
University in 2001 and an M.S. in engineeringfrom Nara Institute of Science and Technology in
2003. He joined NTT in 2003. His research interests include a wide range of topics related to
SysML (Systems Modeling Language) such as
distributed training, efficient inference runtime,scalable surveillance applications, and theories
for deep learning. He is a member of the Information Processing Society of Japan and the Association for Computing Machinery.

Masashi Toyama

Engineer, Distributed Data Processing Platform SE Project, NTT Software Innovation
Center.
She received a B.E. and M.E. in computer science from Nagoya University, Aichi, in 2011 and2013. She joined NTT in 2013. Her researchinterests include deep learning and software
engineering.

Taku Sasaki

Senior Research Engineer, Distributed Data
Processing Platform SE Project, NTT Software
Innovation Center.
He received a B.S. and M.S. in information andcomputer science from Keio University, Kanagawa, in 2003 and 2005. He joined NTT in 2005.
His current research interests include data science and software engineering.

Engineer, Distributed Data Processing Platform SE Project, NTT Software Innovation
Center.
He received a B.S. and M.S. from Tokyo Institute of Technology in 2014 and 2016. He joined
NTT Software Innovation Center in 2016. His
current research interests include attention-based
deep learning and computer vision.

Keita Mikami

Shin’ya Yamaguchi

Senior Research Engineer, Distributed Data
Processing Platform SE Project, NTT Software
Innovation Center.
He received a B.S. and M.S. in information andcomputer science from Waseda University,
Tokyo, in 2005 and 2007. He joined NTT in
2007. His current research interests include data
science and software engineering. He is a member of the Information Processing Society ofJapan.

Researcher, Distributed Data Processing Platform SE Project, NTT Software Innovation
Center.
He received a B.E. and M.E. from Yokohama
National University, Kanagawa, in 2015 and2017. He joined NTT Software Innovation Center in 2017. His research interests include deep
(machine) learning, particularly transfer learning, representation learning, and deep generativemodels.

Yutaka Hirokawa

Katsuo Inaya

Engineer, Distributed Data Processing Platform SE Project, NTT Software Innovation
Center.
He received a B.E. and M.E. in computer science from Tohoku University, Miyagi, in 2003
and 2005. He joined NTT in 2005. His researchinterests include anomaly network traffic detection.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Senior Research Engineer, Supervisor, Distributed Data Processing Platform SE Project, NTT
Software Innovation Center.
He joined NTT in 1995. He is an experienced
engineer with a long history of working in theinformation technology and services industry. He
has experience in the areas of enterprise software, business development, strategy, strategic
partnerships, and mobile devices. His current
research interests include deep learning.

55

Regular Articles

Highly Efficient Gate Controllabilityof Rashba Spin-orbit Interaction in a
Gate-all-around InAs Nanowire
MOSFET
Keiko Takase, Guoqiang Zhang, Kouta Tateno,
and Satoshi Sasaki
Abstract

Indium arsenide (InAs) is a III-V semiconductor with high mobility that has attracted much attention
as a next-generation semiconductor to replace the current silicon technology. At the same time, InAs is
known to have a large spin-orbit interaction that causes electron spin precession even without an external
magnetic field. Here, we report on an InAs nanowire metal-oxide-semiconductor field-effect transistor(MOSFET) with a gate-all-around structure, where the gate electrode is arranged coaxially with the InAs
channel. We not only show that our device has good FET properties such as a high ON/OFF ratio andhigh mobility but also demonstrate that the Rashba spin-orbit interaction is largely controlled with asmall gate voltage range, with its efficiency being ten times larger than those obtained for previously
reported III-V semiconductor MOS and Schottky FETs. This indicates that our FET is promising as aprototype of a low-power-consumption spin FET, in which transistor ON/OFF is controlled as aconsequence of electron spin flips.
Keywords: nanowire, spin-orbit interaction, spin FET

1. Introduction
Group III-V semiconductors such as indium arsenide (InAs) and InGaAs (indium gallium arsenide)
have been extensively studied as next-generation
high-mobility semiconductors that can replace thecurrent silicon (Si) technology. Another direction ofresearch on III-V semiconductors that has been offundamental interest for decades is their large spinorbit interaction, which acts as an effective magnetic
field and thus can rotate electron spin even in theabsence of an external magnetic field.
As these two research directions merged, a conceptof the spin field-effect transistor (FET) was proposedby Datta et al. in 1990 [1]. In contrast to a normaltransistor, in which transistor ON/OFF is defined by
NTT Technical Review

Vol. 17 No. 2 Feb. 2019the changing electrical current using gate voltage, aswitch for a spin FET relies on reversed spin directionin the channel, which is controlled via the spin-orbit
interaction using gate voltage. The required key effectis the Rashba spin-orbit interaction, which is proportional to the applied electric field. With this interaction, the spin precession can be accessed by the gate
voltage, and the device can consequently be used as aspin transistor.
A spin FET is conceptually shown in Fig. 1. The
spin FET is composed of ferromagnetic electrodes
and an electron channel formed in a semiconductorthat has a strong spin-orbit interaction. When spin is
injected from one of the ferromagnetic electrodes androtated by π via the spin-orbit interaction, the spin
becomes antiparallel to the spin on the other side of56

Regular Articles

Ferromagnetic
electrode

Gate metal rotation

Large
spin-orbit
interaction

2 rotation

Antiparallel[off]

Parallel[on]Fig. 1. Concept of a spin FET. The spin flip in the channel is controlled by the gate voltage via spin-orbit interaction.

the ferromagnetic electrodes, indicating that electrical current is no longer flowing.
In contrast, when the electron spin rotates up to 2π
due to the varied spin-orbit interaction, the spin turns
so it is parallel to the spin in the other electrode, indicating that electron current has started to flow. Thistransistor operation resulting from spin precession
can be realized by changing the gate voltage if we canuse the Rashba spin-orbit interaction.
This type of spin FET is expected to provide many
benefits in engineering as well as to continue to be offundamental interest. For instance, we can expect toachieve a lower-power-consumption transistor when
the spin flip voltage is lower than the pinch-off voltage in normal transistors. In addition, when the spinorbit interaction is large enough, we can integrate thespin FETs using the present Si technology.
Based on this background, various III-V semiconductors have been investigated to examine whether
spin-orbit interaction is indeed controlled by the gate
voltage. Nitta et al. demonstrated for the first timethat the spin-orbit interaction is varied by the gate
voltage using two-dimensional electron gas formed
in a III-V semiconductor quantum well [2]. Later,
Koo et al. demonstrated that spin injection and gate
control of the spin-orbit interaction occurred simultaneously for the same device [3]. Moreover, muchsmaller devices have been developed from quantum
wire [4] and bottom-up nanowires [5–9] with the aimof achieving lower power consumption and high integration. However, the Schottky FETs and metalNTT Technical Review

Vol. 17 No. 2 Feb. 2019oxide-semiconductor field-effect transistors (MOSFETs) used in the previous reports have not greatly
improved the gate controllability of spin-orbit interaction.
In this article, we report that a gate-all-around(GAA) InAs nanowire MOSFET that we recently
developed has superior gate controllability of thespin-orbit interaction, which is ten times higher thanthat obtained for MOSFET or Schottky FETs [10].
Our demonstration will open the way toward developing low-power-consumption spin FETs.
2. Experimental
The GAA InAs nanowire is schematically shown in
Fig. 2(a). The InAs nanowire was grown using thevapor-liquid-solid method [11]. The diameter of the
nanowire is about 100 nm, and the gate length isabout 3.3 µm. The GAA structure was prepared asfollows. We first grow Al 2O 3/HfO 2 (aluminium
oxide: 2 nm/hafnium oxide: 4 nm) as a gate insulator
around the nanowire and then put it on a pre-patterned
SiO2 (silicon dioxide)/Si substrate and deposit thegate metal by an electron beam. A coaxial gate structure using a thin high-k gate insulator enables us toapply a strong electric field to the nanowire channel.
The details of the fabrication procedure are describedin other reports [10, 12, 13].
A TEM (transmission electron microscope) imageof our device is shown in Fig. 2(c). We can see that athin gate insulator and surrounding gate metal are57

Regular Articles(a)Gate insulator

(b)

(c)

InAs
NW

D

InAs

Metal

G
InAs
nanowire

SiO2

Vg

S
Vsd

(d)

Au
Ti
HfO2
Al2O3

(e)

50 nm

(f)

(g)

(h)

(i)

50 nm
Al: aluminium
Al2O3: aluminium oxide
As: arsenide
Au: gold
Hf: hafnium

HfO2: hafnium oxide
In: Indium
InAs: Indium arsenide
NW: nanowire
SiO2: silicon dioxide

Ti: titanium
Vg: gate voltage
Vsd: source-drain voltage

Fig. 2. (a) Schematic illustration of our GAA InAs nanowire MOSFET. (b) Schematic cross section of our device. (c) TEM(transmission electron microscope) image of the cross section of our device. (d)–(i) EDS (energy dispersive X-ray
spectroscopy) images taken at the same time as image shown in (c). These images correspond to each element in
the device.

formed around the nanowire. These geometries wereconfirmed by energy dispersive X-ray spectroscopy,
which also indicates that there are no notable impurities or migration. The measurements were performedat temperatures ranging from room temperature to
1.5 K.
3. Results and discussion
In this section, we report the results of our experiments and explain their significance.
3.1 FET characteristics
The transfer and output characteristics obtained atroom temperature and 1.5 K are presented inFig. 3(a)–(d). As shown in Fig. 3(a) and (c), the electrical current increases rapidly as the gate voltage
increases, indicating that the device is acting as atransistor. The ON/OFF ratios extracted from thesedata are 104 and 105 at room temperature and 1.5 K.
The subthreshold swing defined as SS ≡ dVg/dlog(Isd)is 350 mV/dec and 25 meV/dec at room temperatureand 1.5 K. The mobility ranges from 400 to 1200
cm2/Vs. These values are comparable to those reported for back-gated InAs nanowires fabricated by other
NTT Technical Review

Vol. 17 No. 2 Feb. 2019groups [14–16]. This demonstrates that our GAA
InAs nanowire device shows good FET performance.
3.2 Gate controllability of spin-orbit interaction
We next investigate the strength of the spin-orbit
interaction and its gate controllability. For this aim,
we measure magnetoconductance to examine whether weak localization and weak antilocalization effects
are observed for our device. Weak localization (antilocalization) is a quantum correction effect, whichhappens in the absence (presence) of the spin-orbit
interaction. When the spin-orbit interaction is negligibly small, time-reversal symmetry is protected atzero magnetic field, and thus, electron waves propagating in the clockwise and anticlockwise directions
produce constructive interference, leading to a weak
localization. Then when the magnetic field is applied,
time-reversal symmetry is broken, and the conductance is increased with the increasing magnetic field.
In contrast, the opposite behavior occurs when the
spin-orbit interaction is strong. At zero magnetic
field, the spin flips due to the spin-orbit interaction,and thus, destructive interference occurs rather thanconstructive interference. This is known as a weak
antilocalization effect, which results in decreasing
58

Regular Articles

20

10

10

dec

10−1

SS

10−2
10−3
−2

0
−2

(c)

−1

0
1
Vg (V)

1.5 K

2.5

1.0

2

(d)

Vsd = 50 mV
40
30
20
10

2.0
1.5

2

0
1
−1
Gate voltage Vg (V)

0.5

1
10−2
10−4
10−6
−2

0.0
−2

−1

1
0
Vg (V)

−1

2

0

RT

Vg = 2 V
1.6
1.2
0.8

40
30
20
10
0
0.0

0.1
0.2
0.3
0.4
Source-drain voltage Vsd (V)0.5

Vg = 2 V
1.6
1.2
0.8

15 1.5 K

10102
lsd (µA)

Source-drain current lsd (µA)

1

Source-drain current lsd (µA)4030(b) 50Vsd = 500 mV
400
300
200
100

Source-drain current lsd (µA)

RT

lsd (µA)

Source-drain current lsd (µA)(a) 50

1

2

5

0
0.0

0.1

0.2

0.3

0.4

0.5

Source-drain voltage Vsd (V)Gate voltage Vg (V)RT: room temperature
SS: subthreshold swing

Fig. 3. (a), (c) Transfer characteristics and (b), (d) output characteristics, which were obtained at room temperature and
1.5 K.

magnetoconductance with the increasing magnetic
field.
Such weak localization and antilocalization effects
were first considered in two-dimensional electron
systems by Hikami et al. [17] and are now observedin various two-dimensional materials that possess
strong spin-orbit interaction. Moreover, these effects
are also observed with one-dimensional systems. In
disordered one-dimensional systems, magnetoconductance (defined here as the difference ∆G from thezero-field conductance) is described with the following equation:

[(
(

2
2
∆G = − 2e 3 12 + 4 2 + W 4
3lso
hLg 2 lφ
3lB2
− 1 12 + W 4
2 lφ
3lB

) ]
−1/2

,

)

−1/2

(1)

where h is Planck’s constant, Lg is the gate length, lφ
is the phase relaxation length, lso is the spin-orbit
length, W is the nanowire diameter, and lB is the magnetic length given by lB = h/(2πeB) (e: electron
charge, B: magnetic field).
The data are analyzed using this model. In Fig. 4(a),NTT Technical Review

Vol. 17 No. 2 Feb. 2019

∆G is shown as a function of B for various values ofgate voltage Vg. We can see that ∆G with respect to B
shows a dip-to-peak structure with increasing gate
voltage, corresponding to the transition from weak
localization to weak antilocalization. Moreover, themean free path obtained for our device is about 10 nm
and is much smaller than the nanowire diameter ofapproximately 100 nm, so the system indeed satisfies
the condition in which the model can be applied. Thelines in Fig. 4(a) indicate that our data fit nicely withthe one-dimensional model described by Eq. (1).
From this fitting, we can extract lso and lφ, which areplotted as a function of Vg as shown in Fig. 4(b). This
clearly shows that lso varies significantly with Vg andthat lso becomes smaller than lφ around the crossoverfrom weak localization to weak antilocalization.
3.3 Comparison with previous reports
Next, we compare our results with those obtainedfor previously reported III-V semiconductors withvarious sample geometries. The dependence of lso on
Vg measured for various InAs nanowires is summarized
in Fig. 5(a). Our results plotted in different colors(blue and orange) are obtained for the same sample
59Regular Articles(a) 1.5(b)
0.23 V

0.28 V

0.0

0.36 V

−0.2

0.5

lso
l

400

0.2

0.59 V
0.88 V

0.00

0.03
B (T)

0.28 V

0.06

0.0
0.59 V
0.88 V−0.5−1.0

1.47 V−0.10 −0.05 0.00

0.05

0.10

Magnetic field B (T)Spin-orbit length lso
Phase relaxation length l (nm)Magnetoconductance G (µS)

1.0

G (µS)

0.4

300

200

100

0
0.0

0.5

1.0

1.5

Gate voltage Vg (V)Fig. 4. (a) Magnetoconductance with respect to magnetic field for different gate voltages. (b) Spin-orbit length and phaserelaxation length as a function of gate voltage.

with different cooling times, which confirmed therobustness of the sample as well as reproducibility
after thermal cycles. We can see that there is little
modulation in lso for InAs nanowires with conventional back gate or top gate structures [5–9], even if agate voltage over 10 V is applied.
In contrast, our GAA and the ion-gated device
developed by Liang et al. [18] show large modulationin lso within a much smaller Vg range. This is because
the GAA and ion-gated structures can change theelectric field over a wide range, indicating that the
Rashba spin-orbit interaction, which is proportional
to the electric field, varies widely with a small gate
voltage.
We then deduce the Rashba coupling parameter aRfrom lso. Here, aR is defined in the Hamiltonian representing the Rashba spin-orbit interaction,
H = aR σ · (p × ẑ) (h̄: Planck’s constant divided byh̄
2π, p: momentum, ẑ: direction parallel to the electricfield). We can intuitively understand that an effective
magnetic field, which occurs in the direction perpendicular to the momentum and electric field, interacts
with spin; consequently, the strength of the spin-orbit
interaction is determined by the size of aR. Here, aRh̄2(m*:can relate with lso in the formula, aR =2m*lsoeffective mass).
In Fig. 5(b), aR obtained for our device is comNTT Technical Review

Vol. 17 No. 2 Feb. 2019

pared with those reported for one-dimensional andtwo-dimensional FETs that have been investigated in
the context of spin FETs. The gate modulation efficiency of aR for our device is more than ten times
larger than those obtained for back-gated InAs
nanowire [5] and Schottky FETs fabricated fromvarious III-V semiconductor quantum wells [2, 3].
Although our GAA MOSFET is a bit less effective
than the ion-gated device, our device has great significance in that it demonstrates that such high controllability and large Rashba parameter are obtainedusing a MOSFET, which is much faster and morestable than ion-gated devices.
We finally note the electric field ER associated with
the Rashba spin-orbit interaction, which is given by
ER = aR/(a0e) (a0: Rashba coefficient, a0 = 1.17 nm2for InAs [19]). We obtain the ratio to the peak electricfield E0 that is expected from the gate geometry. The
results of ER vs. E0 obtained for our GAA device andthe ion-gated device are plotted in Fig. 5(c). With
both devices, ER is reduced more than E0 is. The ratio
stays at 5% for the entire Vg range (Fig. 5(d)). This
indicates that the screening effect due to the interface
states that may be included in the devices is nearly thesame for both devices.
4. Summary
We demonstrated that our GAA device has higher
60

Regular Articles(a) 400
Spin-orbit length lso (nm)Back gate MOS

GAA MOS

350

This work

Hansen et al.
Dhara et al.
Liang et al.
Roulleau et al.
Scherübl et al.

Ion gate
Liang et al.

300
250
200
150
100
50
0

0

2

4

6
8
Gate voltage Vg (V)(c)

3

Ion gate
3

GAA MOS(This work)

2

Back gate

0

1

0.5

1.0

1.5

2.0

0
2.5

2.0
1.0
0.0

0

10 20 30 40 50 60
Peak electric field E0 (×107 V/m)
Ion gate
This work

0.06
0.03
0.00
0.0

Gate voltage Vg (V)14

Ion gate
This work

0.09

InGaAs QWInAs 2DEG

0.0

12

(d)

ER/E0

1

2

3.0

ER (×107 V/m)

4

ER (×107 V/m)

Rashba parameter R (×10−11 eVm)

(b)

10

0.4

0.8

1.2

1.6

Gate voltage Vg (V)QW: quantum well
2DEG: two-dimensional electron gas

Fig. 5. (a) Comparison of dependence of lso on Vg for our device and those reported previously [5–9]. (b) Comparison of the
Rashba parameter vs. gate voltage for our device and previously reported III-V semiconductor devices [2, 3, 5, 18].
(c) ER as a function of E0 for our device and the ion-gated device reported by Liang et al. [18]. (d) ER/E0 ratio as afunction of Vg for the devices in (c).

gate controllability of the spin-orbit interaction thanthose obtained for standard MOS or Schottky FETs
fabricated from one- and two-dimensional III-V
semiconductors. The biggest advantage is that highgate controllability and the large Rashba spin-orbit
interaction are obtained at a small gate voltage usinga MOSFET structure. This opens the way towardrealizing nanoscale spin FETs.

References
[1]
[2]
[3]
[4]
[5]
[6]

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

S. Datta and B. Das, “Electronic Analog of the Electro-optic
Modulator,” Appl. Phys. Lett., Vol. 56, No. 7, pp. 665–667, 1990.
J. Nitta, T. Akazaki, H. Takayanagi, and T. Enoki, “Gate Control of
Spin-orbit Interaction in an Inverted In 0.53Ga0.47As/In0.52Al0.48As
Heterostructure,” Phys. Rev. Lett., Vol. 78, No. 7, 1335, 1997.
H. C. Koo, J. H. Kwon, J. Eom, J. Chang, S. H. Han, and M. Johnson,“Control of Spin Precession in a Spin-injected Field EffectTransistor,” Science, Vol. 325, 1515, 2009.
T. Schäpers, J. Knobbe, and V. A. Guzenko, “Effect of Rashba Spinorbit Coupling on Magnetotransport in InGaAs∕InP Quantum Wire
Structures,” Phys. Rev. B, Vol. 69, 235323, 2004.
A. E. Hansen, M. T. Björk, C. Fasth, C. Thelander, and L. Samuelson,“Spin Relaxation in InAs Nanowires Studied by Tunable Weak
Antilocalization,” Phys. Rev. B, Vol. 71, 205328, 2005.
S. Dhara, H. S. Solanki, V. Singh, A. Narayanan, P. Chaudhari, M.
Gokhale, A. Bhattacharya, and M. M. Deshmukh, “Magnetotransport

61Regular Articles

Properties of Individual InAs Nanowires,” Phys. Rev. B, Vol. 79,
121311, 2009.
[7] S. Estévez Hernández, M. Akabori, K. Sladek, C. Volk, S. Alagha, H.
Hardtdegen, M. G. Pala, N. Demarina, D. Grützmacher, and T.
Schäpers, “Spin-orbit Coupling and Phase Coherence in InAs
Nanowires,” Phys. Rev. B, Vol. 82, 235303, 2010.
[8] P. Roulleau, T. Choi, S. Riedi, T. Heinzel, I. Shorubalko, T. Ihn, and
K. Ensslin, “Suppression of Weak Antilocalization in InAs
Nanowires,” Phys. Rev. B, Vol. 81, 155449, 2010.
[9] Z. Scherübl, G. Fülöp, M. H. Madsen, J. Nygård, and S. Csonka,“Electrical Tuning of Rashba Spin-orbit Interaction in Multigated
InAs Nanowires,” Phys. Rev. B, Vol. 94, 035444, 2016.
[10] K. Takase, Y. Ashikawa, G. Zhang, K. Tateno, and S. Sasaki, “Highly
Gate-tuneable Rashba Spin-orbit Interaction in a Gate-all-around
InAs Nanowire Metal-oxide-semiconductor Field-effect Transistor,”
Sci. Rep., Vol. 7, 930, 2017.
[11] K. Tateno, G. Zhang, and H. Nakano, “Growth of GaInAs/AlInAs
Heterostructure Nanowires for Long-wavelength Photon Emission,”
Nano Lett., Vol. 8, No. 11, pp. 3645–3650, 2008.
[12] S. Sasaki, K. Tateno, G. Zhang, H. Suominen, Y. Harada, S. Saito, A.
Fujiwara, T. Sogawa, and K. Muraki, “Encapsulated Gate-all-around
InAs Nanowire Field-effect Transistors,” Appl. Phys. Lett., Vol. 103,
213502, 2013.
[13] S. Sasaki, K. Tateno, G. Zhang, H. Pigot, Y. Harada, S. Saito, A.

Fujiwara, T. Sogawa, and K. Muraki, “Self-aligned Gate-all-around
InAs/InP Core–shell Nanowire Field-effect Transistors,” Jpn. J. Appl.
Phys, Vol. 54, No. 4S, 2015.
[14] K. Storm, G. Nylund, L. Samuelson, and A. P. Micolich, “Realizing
Lateral Wrap-gated Nanowire FETs: Controlling Gate Length with
Chemistry Rather than Lithography,” Nano Lett., Vol. 12, No. 1, pp.
1–6, 2012.
[15] S. Dhara, S. Sengupta, H. S. Solanki, A. Maurya, A. Pavan R., M. R.
Gokhale, A. Bhattacharya, and M. M. Deshmukh, “Facile Fabricationof Lateral Nanowire Wrap-gate Devices with Improved Performance,”
Appl. Phys. Lett., Vol. 99, 173101, 2011.
[16] Q. Li, S. Huang, D. Pan, J. Wang, J. Zhao, and H. Q. Xu, “Suspended
InAs Nanowire Gate-all-around Field-effect Transistors,” Appl. Phys.
Lett., Vol. 105, 113106, 2014.
[17] S. Hikami, A. I. Larkin, and Y. Nagaoka, “Spin-orbit Interaction andMagnetoresistance in the Two Dimensional Random System,” Prog.
Theor. Phys., Vol. 63, No. 2, pp. 707–710, 1980.
[18] D. Liang and X. P. A. Gao, “Strong Tuning of Rashba Spin-orbit
Interaction in Single InAs Nanowires,” Nano Lett., Vol. 12, No. 6, pp.
3263–3267, 2012.
[19] R. Winkler, “Spin-orbit Coupling Effects in Two-dimensional
Electron and Hole Systems,” Springer–Verlag, Berlin, Heidelberg,
New York, 2003.

Keiko Takase

Kouta Tateno

Senior Research Scientist, NTT Basic ResearchLaboratories.
She received a B.S., M.S., and Ph.D. in physics
from the University of Tokyo in 2004, 2006, and2009. During 2006–2009, she was a research
fellow (DC1) of the Japan Society for the Promotion of Science (JSPS). She joined NTT Basic
Research Laboratories in 2009. Since then, she
has been studying electrical transport in lowdimensional semiconductors. She is a member of
the Physical Society of Japan (JPS).

Guoqiang Zhang

Senior Research Scientist, NTT Basic ResearchLaboratories.
He received a B.S. in electronic ceramic materials science and an M.S. in semiconductor physics and chemistry from Zhejiang University, P. R.
China, in 1997 and 2000, and a Ph.D. in electronic materials science from Shizuoka University, Japan, in 2004. He joined NTT Basic
Research Laboratories in 2006. Since then, he
has been working on the growth and characterization of nanowire-based nanostructures. He is amember of the Japan Society of Applied Physics(JSAP) and the Japanese Association for Crystal
Growth.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Senior Research Scientist, NTT Basic ResearchLaboratories.
He received a B.S., M.S., and Ph.D. in chemistry from the University of Tokyo in 1991, 1993,
and 2001. He joined NTT Opto-electronics Laboratories in 1993. His current research interests
include the fabrication and physics of nanostructures using semiconductor nanowires. He is amember of JSAP.

Satoshi Sasaki

Senior Research Scientist, NTT Basic ResearchLaboratories.
He received a B.S., M.S., and Ph.D. in applied
physics from the University of Tokyo in 1988,
1990, and 1993. He joined NTT Basic ResearchLaboratories in 1993. Since then, he has been
studying transport properties of mesoscopic
devices, especially quantum dots. He is a member of JPS and JSAP.

62

Global Standardization Activities

New Standardization Trends at
GlobalPlatform—Secure Components
for the IoT Era
Eikazu Niwano
Abstract

GlobalPlatform, which was founded as an international standardization organization targeting smart
cards, has expanded its scope of standardization to include a variety of Secure Elements, Trusted Execution Environments, and devices. Furthermore, in the light of current trends surrounding the Internet ofThings (IoT) and ecosystems, GlobalPlatform is broadening the functions it provides from application
management to device trust management. This article introduces new initiatives toward the IoT andecosystem era at GlobalPlatform, which has recently restructured its organization.
Keywords: IoT security, tamper resistant module, secure chip

1. GlobalPlatform
Smart cards have elements similar to those of acomputer such as a CPU (central processing unit) andmemory and are secure components with hardware
characteristics robust to external attacks, making
them tamper resistant. They can be used for security
processes and safe data management as in user
authentication and data encryption.
Smart cards are presently being used in many fields
including payment systems (smart credit cards etc.),
communications (subscriber identity module (SIM)cards), transportation (Suica train/bus cards etc.),
workplace/access management (employee identification (ID) cards), and public services (electronic passports, My Number (social security and tax number)cards, etc.).
GlobalPlatform [1] was founded in 1999 as aninternational standardization organization targetingthe management of application programs embedded
within smart cards (Fig. 1). It is the world’s leadingindustry standardization organization in the field ofsecure components that enable remote and secureloading/managing of applications in smart cards via
collaboration between smart card issuers and service
providers after smart card issuance [2].
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Since its founding, GlobalPlatform has expanded
the range of secure components targeted for standardization from smart cards to various types of Secure
Elements (SEs) [3], including embedded SIMs(eSIMs), and to Trusted Execution Environments(TEEs) [4] within a device independent of the device
operating system (OS). Of particular importance hereis that GlobalPlatform is now extending its target ofmanagement as far as the device level. In addition, its
scope of standardization continues to expand toinclude the updating of SE firmware and the creationof an SE/TEE certification program.
Consequently, in addition to contributions fromtraditional SE-related operators such as chip and card
manufacturers for smart card systems as well as telecom operators and others, proactive contributions
from mobile-device chip manufacturers such as ARM
and mobile-device-related operators such as Apple,
Samsung, and Qualcomm in recent years havebecome an important trend. At present, about 100
companies from around the world are participating in
GlobalPlatform as stakeholders.
Meanwhile, a wide variety and massive number of
low-to-high-end Internet of Things (IoT) devices arecoming to be dynamically added, connected, coordinated, and controlled in ever-expanding IoT environments
63

Global Standardization Activities

Board of Directors

Operations Secretariat
Tono Aspinal– Rob Coombs, ARM– Nils Gerhardt, G+D Mobile Security– Stephanie El Rhomri, FIME
– Remi De Fouchier, Gemalto– Eikazu Niwano, NTT– Sebastien Hans, Oracle– Jeremy O’Donoghue, Qualcomm– Mark Lipford, Sprint– Olivier Van Nieuwenhuyze, STMicroelectronics– Christophe Colas, Trustonic– Marc Kekicheff, VISA

Technical Director
Gil Bernabeu

Executive Director
Kevin Gillick

Functional Certification
Secretariat
Security Certification
Secretariat

Technical committees

Advisory Council
Marketing Task Forces

Regional Task Forces

Consumer IoT TF
Nils Gerhardt,
G+D Mobile Security

China TF
Vicky Guo, CAICT

Industrial IoT TF
Sebastien Hans, Oracle

Japan TF
Eikazu Niwano, NTT

Security TF
Olivier Van Nieuwenhuyze,
ST Microelectronics

TEE Committee
Christophe Colas,
Trustonic

TPS Committee
Jeremy O’Donoghue,
Qualcomm

Multi SE WG

TEE Compliance
WG

Functional APIs
WG

SE Security WG

TEE Specification
WG

Low Level
Services WG

SE Compliance
WG

TEE Security
WG

Root of Trust
Services WG

SE Committee
Karl Eglof Hartel,
G+D Mobile Security

SE Specification
WG

API: application programming interface
SE: Secure Element
TEE: Trusted Execution Environment

TF: Task ForceTPS: Trusted Platform Service
WG: Working Group

Fig. 1. GlobalPlatform organizational structure.

that drive the formation of ecosystems.
With the appearance of such complex and dynamic
environments, much evidence and many incidents inrelation to cyber-attacks are starting to be reported,
revealing the vulnerability of IoT devices. These
include the hijacking of a connected car and the malicious manipulation of its steering wheel, unauthorized operation of medical equipment, and DDoS
(distributed denial of service) attacks using home
digital cameras as a springboard.
Consequently, much attention is being focused on
how to guarantee device trustworthiness as a criticalissue, such as in determining the authenticity of thehuge variety of interconnected IoT devices.
Against this background, GlobalPlatform seeks toenable a wide range of device manufacturers anddigital service providers to collaborate in performing
highly reliable and secure device management, and to
this end, it is working to expand the role of securecomponents from user authentication to device
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

authentication.
Studies have therefore begun on IoT/device authentication at the requirements level, and in conjunction
with the first reorganization of technical committees
since the founding of GlobalPlatform in 1999, theTrusted Platform Services (TPS) Committee has been
established to decide on technical specifications fordevice authentication.
2. Advisory Council
GlobalPlatform has established an Advisory Council as a forum for discussing requirements before
formulating technical specifications. At present, theAdvisory Council consists of market and regional
task forces.
There are three market task forces: Consumer IoT,
Industrial IoT, and Security. These task forces holddiscussions on issues respectively concerning theconsumer IoT field such as wearable devices, the64Global Standardization Activities

industrial IoT field such as Industry 4.0, and Root ofTrust (RoT), Chain of Trust, and support for the latest
encryption schemes (diverse and advanced encryption schemes, lightweight encryption schemes for
specific devices, etc.).
The regional task forces, meanwhile, are established considering where GlobalPlatform members
are based. The regional task forces share GlobalPlatform information and regional circumstances, study
regional conditions and deployment issues, and holddiscussions on region-originating technology. At
present, there are two regional task forces: China andJapan.
3. Technical committees
The GlobalPlatform bodies that decide on technicalspecifications are the technical committees. Somename changes of these committees have occurred inorder to reflect the state of study concerning securecomponents, which continue to expand from smart
cards. The Card Committee is now the SE Committee, and the Device Committee is now the TEE Committee. Additionally, as mentioned above, the mostnoteworthy change here is the establishment of the
TPS Committee to study the trustworthiness of devices. Here, the study theme is the trustworthiness ofdevices that use secure components.
Through these committees, GlobalPlatform aims tofacilitate collaboration between device manufacturers and digital service providers and provide secure
digital services regardless of the market sector or
device type. To this end, it has technologies forauthentication, connectivity, privacy, and security
that enable it to provide the following services:(1) Protection of digital services(2) Secure remote management of digital services(3) Certification of secure components
4. Protection of digital services
Protection of digital services implements GlobalPlatform’s technology designed to protect digital
services and assets that use secure components. It
provides protection of digital assets (credentials suchas fingerprints and authentication/encryption keys)
and associated security services (authentication etc.).
This protection is achieved by standardizing two
types of secure component technologies—SE andTEE—supporting diverse market needs, as described
below.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

4.1 SE
This is a secure element consisting of tamperresistant hardware such as a smart card. It includes
smart cards and USB (universal serial bus) tokens
that link with a terminal from the outside through areader or writer. Another type of SE includes SIM,
smartSD (Secure Digital memory card), and embedded SEs (eSIM, embedded universal integrated circuit cards (eUICCs), etc.) used inside mobile devices,
as well as integrated SE (iSE) integrated with a SoC(system on chip).
Standardization of access to multiple SEs within amobile device is also progressing. Here, an Open
Mobile API (OMAPI) is being specified as an application programming interface (API) for accessing an
SE from a mobile application. Furthermore, giventhat mobile devices support proximity communication technology called near-field communication(NFC)*1, GlobalPlatform is also engaged in joint
studies with the NFC Forum and the European Telecommunications Standards Institute (ETSI) on Host
Card Emulation as an emulation environment targeting mobile-device cards that use NFC and on Contactless Card Emulation Environments as a mechanism for accessing various types of SEs. Studies arealso progressing on the updating of SE firmware andspecifically on a mechanism called Virtual Primary
Platform as a new study item that separates the iSE
OS into upper and lower sections as requested by the
GSM Association (GSMA)*2.
At present, about 22 billion GlobalPlatform-conforming SEs have been deployed (41% of the SE
market). It is noteworthy in relation to IoT that
GSMA has released remote provisioning specifications for eUICCs that use GlobalPlatform technologies, and that the automobile industry is adopting thismechanism.
4.2 TEE
In addition to tamper-resistant hardware such as
SEs, GlobalPlatform is also moving forward with thestandardization of TEEs (Fig. 2) to provide a secure
execution environment independent of the existing
OS in devices such as mobile phones and to guarantee
a level of security higher than that of the device OS.
*1 NFC: A wireless communications standard that includes existing
contactless smart card communication standards. It enables theuse of a contactless smart card function, reader/writer function,and inter-device (peer-to-peer) communication function.
*2 GSMA: An industry group made up of mobile communications
operators and related companies that have adopted the GSM mobile phone system.

65

Global Standardization Activities

REE APPLICATION ENVIRONMENT

TRUSTED EXECUTION ENVIRONMENT
TrustedApplication
DRM

TrustedApplication
Payment

TrustedApplication
Corporate

TEE Internal API
TEE
Comm.
Agent

TEE Client API

Rich OS

TrustedCore
Framework

Trusted
Drivers

Trusted OS Components

Hardware keys, storage, TUI peripherals(screen and keyboard), Secure Element

Hardware Platform

Source: GlobalPlatform
DRM: digital rights management
REE: Rich Execution Environment
TUI: trusted user interface

Fig. 2. TEE architecture.

A TEE executes only authorized software known astrusted applications and guarantees end-to-end security such as the protection of privacy and data accessrights.
A TEE includes interfaces with applications, thedevice OS, and SEs and provides a trusted user interface (TUI) environment to prevent a man-in-thebrowser attack.
At present, TEEs are being used for applications
requiring a high level of security such as biometricauthentications, digital rights management (DRM)for video content, and payments. The TEE securecomponent has been mentioned, for example, in IoT
Security Guidelines ver. 1.0 [5], specified by the IoT
Acceleration Consortium,*3 which reflects its ongoing deployment.
4.3 Four features of digital services protection
The following describes four features provided by
the SE and TEE secure components to protect digital
services and resources (Fig. 3).
The first feature is the provision of a secure execution environment isolated from the device OS bymanaging the RoT within the secure component.
The second feature is end-to-end secure execution
NTT Technical Review

Vol. 17 No. 2 Feb. 2019of application management within the secure component through multi-actors (device manufacturers,
digital service providers, etc.). Here, SE firmware isalso targeted as a managed resource.
The third feature is the ability to add and provide
security services common to multiple market sectors
on the same secure component in a flexible manner.
Finally, the fourth feature is the ability of multiple
digital service providers to respectively manage an
isolated and independent area on the same securecomponent.
5. Secure remote management of digital services
Another GlobalPlatform technology is remotemanagement. In an IoT environment, IoT devices
including sensors are installed in all sorts of places,
so on-site management of these devices over the longterm is hardly realistic. A remote management function is extremely important under these circumstances.
*3 IoT Acceleration Consortium: A body established in Japan forthe purpose of constructing a system for developing and demonstrating IoT technologies and creating new business models
through the participation and collaboration of government, industry, and academia.

66

Global Standardization Activities

Isolated environment
where digital service
execution is protected

Protects application
integrity when deployed
alongside other services

Root of Trust(RoT)

Application
Management

Isolation

Flexible
Security

End-to-end security
and privacy from origin
to user delivery

Security requirementsof multiple markets on
same secure component

Source: GlobalPlatform

Fig. 3. Four features of digital services protection.

This technology enables secure end-to-end, remotemanagement of SEs such as eSIMs embedded in IoT
devices.
6. Certification (authentication) of
secure components
GlobalPlatform provides a program for performing
functional certification of SE/TEE products and security certification of TEE products. In this regard,
GlobalPlatform has begun to collaborate with the
FIDO Alliance,*4 an organization that promotes thestandardization of multi-factor authentication including biometric authentication. Discussions have begun
on how to configure and manage FIDO applications(keys, applications) in an internal device environment
including secure components and on the necessity of
security certification in relation to this configuration.
Furthermore, in relation to cybersecurity and the
problem of supply chains for parts in ever-growing
ecosystems, lively discussions are taking place invarious organizations around the world on the importance of having a certification mechanism that
includes hardware. This certification program is
expected to become increasingly important in conjunction with security-by-design principles as the
IoT/ecosystem era arrives.
7. Device protection—Device Trust ArchitectureThis section focuses on device protection as the latest initiative taken up by GlobalPlatform. It is an
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

extremely important key to achieving cybersecurity
considering the future expansion of IoT and ecosystem environments.
Device protection is a mechanism related to device
trustworthiness, including the authentication ofdevice authenticity. Technical specifications fordevice protection will be specified by the TPS Committee.
In relation to device protection, the Trusted Computing Group (TCG)*5 has been standardizing solutions using the Trusted Platform Module (TPM) forsome time as part of trusted computing, but GlobalPlatform seeks to develop IoT devices as a generalpurpose mechanism using secure components.
As stated above, a key feature of secure components is that they enable device manufacturers anddigital service providers to remotely add or update avariety of applications including security functions(and firmware in the case of SEs) regardless of themarket sector or device type. Using secure components in this way enables the provision of services
such as device RoT, chain of trust, and remote device
identification and attestation, which, in turn, enables
trustworthy, safe, and flexible device management ina multi-actor environment [6].
This device protection mechanism is called Device
Trust Architecture (DTA) [7]. The plan is to promotestudies of DTA as architecture integrated with secure*4 FIDO Alliance: A standardization organization targeting multifactor authentication including biometric authentication.
*5 TCG: An organization promoting the study of TPM having a secure boot function.

67

Global Standardization Activities

Trust

DeviceSecure component

Applicationof

Secure applications

App

App

Internal API
Device OS

Execution
Environment

Functional API

Chain

Secure services
RoT services
Boot

RoT

Source: GlobalPlatform

Fig. 4. Device Trust Architecture.

components for protecting digital services (Fig. 4).
8. Future outlook—Toward the IoT era
A major strength of GlobalPlatform is its collaboration with many standardization organizations. In
addition to standardization organizations targeting
conventional fields such as smart cards, finance, andcommunications, it has recently initiated collaboration with GSMA/OneM2M*6 in the IoT field and the
Car Connectivity Consortium (CCC)*7 in the automotive field. Furthermore, in relation to other typesof tamper-resistant hardware, studies are beginningon a mechanism for generalizing secure boots such as
RoT and chain of trust in collaboration with TCG toassure device trustworthiness, and work is beginningon defining the relationship of secure components
with the Hardware Security Module used for key
management and encryption processing in server
systems.
With the coming of the IoT/ecosystem era, thetrend toward collaboration must be promoted evenfurther. In the IoT security field, however, needs differ according to industry and region due to differences in systems and cultural backgrounds, so it will
become increasingly important to analyze the needs
and use cases of individual industries and regions andto form tie-ups with standardization organizations in
NTT Technical Review

Vol. 17 No. 2 Feb. 2019each of those industries and regions. From here on,standardization and certification through cross-sector
× cross-region efforts as in the case of smart city initiatives will be essential. Collaboration with otherorganizations related to tamper-resistant hardware
according to such industry/regional characteristics
will also be important.
Furthermore, in terms of technology, given an environment that dynamically and dispersedly connects
such an array of stakeholders (developers, operators,
service providers, users) and various types of devices
and systems (and their constituent elements), researchand development and standardization will be required
for an ID component that manages ID/entity information. Key themes will be how to structure and arrange
an ID component using a secure component, how tomanage and certify authenticity/trustworthiness of ID
components, and what mechanism to use to evaluate
and authenticate ID components.
Finally, NTT to date has made contributions to the
management of secure components using the public
key infrastructure scheme. Going forward, we plan tomake further contributions to the IoT security andcybersecurity fields through the application of such
*6 OneM2M: A joint project and standardization organization
formed to promote the standardization of IoT platforms.
*7 CCC: An industry group working on the standardization of in-vehicle smartphone connections.

68Global Standardization Activities

distributed security technologies.

[3]
[4]

References

[5]

[1]
[2]

Website of GlobalPlatform, https://globalplatform.org/
E. Niwano and H. Goromaru, “Standardization Trends at GlobalPlatform,” NTT Technical Review, Vol. 4, No. 11, pp. 48–52, 2006.
https://www.ntt-review.jp/archive/ntttechnical.php?contents=
ntr200611048.pdf

[6]
[7]

GlobalPlatform, “Introduction to Secure Elements,” May 2018.
GlobalPlatform, “Introduction to Trusted Execution Environments,”
May 2018.
IoT Acceleration Consortium, Ministry of Internal Affairs and Communications, Ministry of Economy, Trade and Industry, “IoT Security
Guidelines ver. 1.0,” 2016.
GlobalPlatform, “Deploying and Protecting Digital Services with
Chains of Trust,” May 2018.
GlobalPlatform, “Introduction to Device Trust Architecture,” July
2018.

Eikazu Niwano

NTT Research Professor, NTT Secure Platform Laboratories.
He received a B.S. and M.S. in mathematics
from Waseda University, Tokyo, in 1987 and1989. He joined NTT in 1989 and has studied
distributed system architecture including mobile(messaging)/agent/ubiquitous systems, secure
chips, and social information platforms withinformation security. During 2002–2005, he was
the general manager of the European office of the
NTT laboratories in Paris, and during 2008–2017
he served as producer at the Research and Development Planning Department of NTT. He is currently studying and planning strategies for IoT
security, with a focus on secure components.
He received the Information and Communication Technology Award (the Minister of Internal
Affairs and Communications Award) from The
Telecommunication Committee in Japan in 2018.
He is a member of the board of directors andchairman of the Japan Task Force in GlobalPlatform. He is also a Fellow and member of theboard of directors of the Next Generation Ic Card
System Study group (NICSS) in Japan. He is amember of the Institute of Electrical and Electronics Engineers (IEEE), the Institute of Electronics, Information and Communication Engineers (IEICE), and the Information Processing
Society of Japan (IPSJ).

NTT Technical Review

Vol. 17 No. 2 Feb. 201969

Practical Field Information about
Telecommunication Technologies
Practical Field Information about Telecommunication Technologies

Investigation of the Cause of
Disconnection of Call Parking on a
Business Phone
Technical Assistance and Support Center, NTT EAST
Abstract

Network-interface engineers at the Technical Assistance and Support Center are providing technicalsupport for local maintenance personnel who deal with customers using IP (Internet protocol) services
and public switched telephone network related services. This article describes a problem concerning thedisconnection of call parking in business phones. This is the fiftieth article in a series on telecommunication technologies.
Keywords: call park, business phone, line monitoring system

1. Introduction
Network-interface engineers at the Technical Assistance and Support Center are providing technicalsupport for local maintenance personnel who deal
with customers using Internet protocol (IP) services
such as FLET’S HIKARI NEXT and public switched
telephone network (PSTN)-related services such as
analog lines. This support ranges from conducting
investigations by on-site testing to proposing ways tohandle technical difficulties in order to solve problems whose causes are difficult to identify.
Customers use services in diversified ways, andthus, two approaches are used to investigate thecauses of any technical problems. First, to investigate
the Ethernet segment used by IP services, a method
for analyzing IP packets based on packet capture isutilized. Second, to investigate analog and ISDN
(Integrated Services Digital Network) lines used by
PSTN-based services, a method for analyzing signals
using an oscilloscope and protocol analyzer is utilized. We are also utilizing the tools developed by theTechnical Assistance and Support Center to investigate terminals such as business phones. These tools
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

include the α line monitoring system capable of collecting the unique signals of business phones.
The problem described in this report concerns thedisconnection of call parking in business phones. We
explain how this kind of problem was analyzed byutilizing the above-described analysis methods andthe tools developed by the Technical Assistance andSupport Center.
2. Summary of problem and survey method
The call-park function is used as follows. All
incoming calls on an external line to the customer’s
main phone number are received by a multifunction
telephone installed at the reception desk. After determining who to connect each call to, the receptionist
uses the call-park function to hold the calls and thentransfers the calls to the appropriate staff member,each of whom carries a digital cordless phone.
However, the following event occasionallyoccurred. When an incoming call on an external line
was held using the call-park function to be acceptedby a staff member, the line was put into an active callstate (the line being engaged) while the staff member
70Practical Field Information about Telecommunication Technologies

(1)

LAN
line

Star
wiringOpticalfiber line (4)IP phoneterminal

One-touchconsole LAN line
monitoring
systemCollection of callhistory via web
Cordless phone(extension XXX)

Cordless phone(extension YYY)
*50 units in total

4BRU

LAN

Antennas(50 units)

2OSU

2OSU

LAN

2OSU

LAN

Power-supply hub

Multifunction
telephone atreception desk
(extension AAA)

ONU

LAN

Hikari Denwa
FLET’SHIKARINEXT

WANNX II type-L mainequipment unit (2)(3)

Gigabit-compatible
protocol checker

BRU: broadband router unit
LAN: local area network
ONU: optical network unit star-wiring
monitoring
system

OSU: optical subscriber unit
WAN: wide area network

Fig. 1. Configuration of customer’s equipment.

who should answer the call was being called.
The customer explained that the status of the LED(light-emitting diode) indicator of the call-parking
line on the multifunction telephone (at reception)
when the event occurred changed from blinking (callpark state), to continuously lit (call state), and, after awhile, to unlit (disconnected state). When the statusof the caller of the external-line incoming call where
the problem occurred was checked, it became clearthat the status in which there was no answer to the call
(after the call-park function was initiated) continued
for a while, before the line was eventually disconnected from the caller side.
To conduct normal call operations, the customer set
all digital cordless phones carried by staff members
to respond to all call-park calls via line keys. Local
maintenance personnel checked and/or changed thesetting of the main equipment unit or replaced someterminals, however, these countermeasures did notsolve the problem. Accordingly, the Technical Assistance and Support Center conducted an on-site investigation.
The configuration of the customer’s equipment isshown in Fig. 1. We considered the possibility ofmisoperation or failure of the multifunction teleNTT Technical Review

Vol. 17 No. 2 Feb. 2019

phone and digital cordless phones occurring, so in
our on-site investigation, we added the following fourfunctional elements to the customer’s system: (1)call-park response status monitoring for 50 digitalcordless phones combining an α LAN (local area
network) line monitoring system and IP telephone/
one-touch console; (2) operation status monitoring
for the multifunction telephone (reception) using an
α star-wiring monitoring system); (3) IP packet capture by a gigabit-compatible protocol checker, and (4)records (call history obtained via web) of call information of all calls, including extensions and externallines of the main equipment unit.
To monitor the call-park response status of all 50
digital cordless phones, one IP phone for monitoring
incoming calls on an external line and two one-touch
consoles for monitoring the response status of thedigital cordless phones were newly registered in themain equipment unit, and the terminal response data
were acquired by the α LAN line monitoring system.
The response state of the incoming calls on an external line and the time when a call was incoming can be
determined from the terminal response data of the IP
phone, and the call-park response state and the timethat the line was switched to the call-park state of71

Practical Field Information about Telecommunication Technologies

Table 1. Terminal status according to α LAN line monitoring when the call-park problem occurred.
Time

Terminal

Status (operation etc.)

16:42:31

Extension AAA (reception)

Calling

16:42:36

Extension AAA (reception)

Responding

16:42:50

Extension AAA (reception)

Call parking

16:43:16

Extension XXX (staff member A)

Call-park response

16:44:00

Extension XXX (staff member A)

Call parking

16:44:32

Extension YYY (staff member B)

Call-park response

Remarks
Answer with multifunction telephone
Conveying message to extension XXX while on holdAnswering with digital cordless phone
Talking
Putting on hold once
On hold
Answering with digital cordless phone
Extension YYY: No answer to the call

16:46:37

Calling terminal

Disconnection

16:47:12

Extension YYY (staff member B)

Disconnection

each digital cordless telephone can be understoodfrom the terminal response data of the one-touch consoles. In this way, we constructed a measurement
system to clarify the call-park response state by analyzing these data.
3. Analysis of collected data
We analyzed the data collected in our investigation
as follows.
3.1 α LAN line monitoring system*1
When we checked the park-hold response state of
the 50 digital cordless phones when the park-holdproblem occurred, we confirmed that a different digital cordless phone (staff member) from the destination phone was responding to the park hold. The
operational status of each terminal when the parkhold problem occurred is listed in Table 1. Although
calls transferred from extension AAA (reception) tothe digital cordless phone of extension XXX (staff
member A) are temporarily parked at 16:44:00, at
16:44:32, a park-hold line was responded to with thedigital cordless telephone of extension YYY (staff
member B), that is, not extension XXX. After that,the caller disconnected at 16:46:37, and at 16:47:12,
extension YYY (staff member B) disconnected theline.
3.2 α star-wiring monitoring system*2
The results of checking the data collected by the αstar-wiring monitoring system revealed that the multifunction telephone at reception was correctly operNTT Technical Review

Vol. 17 No. 2 Feb. 2019

Speaking to extension YYYDisconnection~

16:44:34
16:44:50
From the caller to extension YYY

Environmental sound (no response)
16:44:34
16:44:50
From extension YYY to the caller

16:46:37~
16:46:37

Fig. 2. A udio waveform when extension YYY does not
answer the call.

ated to transfer incoming calls via the call-park function.
3.3 Gigabit-compatible protocol checker
An audio-data waveform constructed from packetcapture data when the call-park problem occurred(only after the call-parking line was responded to byextension YYY) is shown in Fig. 2. Although thecaller is speaking to extension YYY, the voice waveform from extension YYY to the caller cannot be
confirmed because there is no verbal response to thecaller’s utterance from extension YYY. The amplitude of the sound was barely confirmed after the*1 α LAN line monitoring system: A monitoring system dedicatedto LAN lines compatible with business phone α series.
*2 α star-wiring monitoring system: A monitoring system dedicatedto star wiring handling the business phone α series. (A monitoring system for bus wiring also exists.)72

Practical Field Information about Telecommunication Technologies

call-park response was confirmed except for a rustling sound—as if something was rubbing against
something else—or the sound of the surrounding
environment.
3.4 Call history obtained via web
We found that the NX II type-L main equipment
unit was operating normally because the call information recorded in the call history provided via web
and the number of calls to and from the external line
clarified from the packet-capture data were in accord.
4. Estimation of the cause of call-park problem
When staff members carry a digital cordless phone
at work, they often put the phone in their breast
pocket; as a result, it is conceivable that the line key
flashing in the call-park state might be accidentally
pushed by another object (e.g., a pen), and the phonemight unintentionally respond to a parked call. For
that reason, it is presumed that the staff member who
responded to a parked call did not notice that thedigital cordless phone was in a call state (accounting
for the rubbing or environmental sound rather than averbal response from extension YYY), and there wasno response to a call from a caller.
5. Countermeasures
Since the customer’s setting was to respond to allparked calls by pressing the line key of all their digitalcordless phones, the above-described unintended
call-park response occurred. Because of the fact thatusers often put their digital cordless phones in theirbreast pockets along with other objects, we proposed
the following three measures to prevent the unintentional call-park response.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

1. Utilization of key lock function
By releasing the key unlock every time an operationis required, it is possible to prevent erroneous
responses to parked calls by unintended button pressing.
2. Utilization of individual on-hold function
By using the individual on-hold function (i.e.,
pushing the hold button) of the multifunction telephone (reception), it is possible to prevent digitalcordless phones other than the destination phonefrom entering a state in which a hold cannot be
responded to.
3. Utilization of park-hold function
When sending an external-line call from the digitalcordless telephone, the customer was pressing theline key to acquire the line and then pressing the destination telephone number. In such an operation,
there is a risk that when a staff member unconsciously presses a line key in an attempt to make an external-line call, he/she will inadvertently respond to aparked call. Therefore, the staff member with a digitalcordless phone uses the park-hold function that isprovided separately from the call park key, it is possible to prevent a false reply to a parked call.
6. Concluding remarks
In the case presented in this report, the problem
concerning the disconnection of call parking in business phones was solved by using a measurement
system utilizing an α LAN line monitoring system tomonitor the operational condition of 50 digital cordless phones. From now onwards, the Technical Assistance and Support Center will analyze the signals anddata using various tools at each interface and contribute to the resolution of problems.

73

Information

Report on NTT R&D Forum 2018
AutumnTakashi Ikebe, Hideo Kato, Atsuyuki Muramoto,
Masaki Hisada, Kentaro Hotta, and Norio Sakaida
Abstract

NTT R&D Forum 2018 Autumn was held at NTT Musashino R&D Center November 26–30, 2018.
(November 26 and 27 were set aside solely for the press and NTT Group employees.) This article provides an overall summary of the forum.
Keywords: R&D forum, smart world, digital transformation

1. Forum overview
The NTT Group, as Your Value Partner, aims toresolve social issues in collaboration with its partners. We believe that being able to resolve social
issues will help to make a smart, digital society a reality. At NTT R&D Forum 2018 Autumn, lectures andexhibits were held on the latest research results
designed to help create this emerging society. The
concept of the forum was Transforming Your Digital
Visions into Reality.
2. Lectures and workshops
On November 28, NTT President and Chief Executive Officer Jun Sawada gave a keynote address entitled “Bringing the Smart World to Life” (Photo 1).
He declared that NTT would work with its partners tobring about the emergence of a smart world. These
efforts are designed to resolve local issues from a
global perspective (thus, glocal activities) throughdigital transformation.
He then introduced current NTT activities in different industries: promotion of the smart city in the Cityof Las Vegas using leading-edge technologies in collaboration with Dell Technologies; smart entertainment, an endeavor with SHOCHIKU Co., Ltd. tointegrate traditional performing arts with information
NTT Technical Review

Vol. 17 No. 2 Feb. 2019and communication technology (ICT); the smart
manufacturing of JSR Corporation, which is improving the productivity of its industrial complex, a venture in which we are partnered with Accenture Japan
Ltd.; and smart mobility, which includes collaboration with Toyota Motor Corporation in the development of basic technologies for automated driving andcollaboration with Nippon Yusen K.K. and MTI Co.,
Ltd. in the development of an Internet of Things (IoT)
platform for the shipping industry.
In connection with the NTT Group’s Medium-Term
Management Strategy called Your Value Partner
2025, President Sawada described the NTT Group’s
global business growth strategy, which has been
designed to strengthen the group’s global competitiveness. He announced that NTT will support its
customers to evolve their businesses, undertake innovative creation, and strengthen and globalize its
research and development (R&D). He ended his
address with a declaration that NTT will make all-out
efforts to achieve its own digital transformation goals
and become Your Value Partner.
The president’s keynote address was followed by aspeech delivered by NTT Senior Vice President, Headof Research and Development Planning, Katsuhiko
Kawazoe (Photo 2). With the title “Making the World
Smart and Technology Natural,” he stated that NTT’s
R&D policy of developing natural technologies was74

Information

Photo 1. K eynote address by Jun Sawada, President andChief Executive Officer, NTT.

Photo 2. K eynote address by Katsuhiko Kawazoe, SeniorVice President, Head of Research andDevelopment Planning, NTT.

motivated by the desire to create a world in whichpeople can engage in human-centric activities while
technologies provide an environment that is friendly
to both people and the earth. As an activity symbolicof this initiative, he announced his plan to launch a
new project called Point of Atmosphere. The aim of
the project is to help create a world in which things
surrounding a person work together autonomously
and interact with the individual in a natural manner.
He then introduced NTT’s activities related to“R&D for a SMART WORLD.” NTT intends to
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Photo 3. S pecial session by Debra Bordignon, Chief
Technology Officer, Dimension Data Australia.

achieve a smart world by working with partners tosolve issues associated with the digital transformation of both society and industry using technologies
that have grown out of its R&D and to generate “theworld’s best, the world’s first, and the world’s amazement.”
Finally, he presented three R&D measures to accelerate innovation: global utilization of R&D results,
globalization of research targets, and establishmentof research organizations outside Japan. He announced
that based on these measures, three laboratories specializing in basic research will be established overseas: NTT Φ Laboratories, which will study quantum
computing science; NTT CIS Laboratories, which
will work on cryptography and information theory;
and NTT MEI Laboratories, which will address biological information processing.
Two special sessions were held at this forum, thefirst on November 29 and the second on November
30. First, Debra Bordignon, Chief Technology Officer, Dimension Data Australia, introduced activities
at the Client Innovation Centre (CIC) designed to create new value with partners in Australia under the titleof “Co-Innovating to Accelerate Transformation andCreate New Value” (Photo 3). In collaboration withDeakin University, FLAIM Systems, and Dimension
Data, CIC incorporated the NTT-developed hitoeTM
material into FLAIM TrainerTM, a firefighter training
simulator that uses virtual reality to enable training ina situation similar to an actual fire. Use of firefighters’ physiological data collected via hitoe has made it
possible to train firefighters effectively without
75

Information

Photo 4. S pecial session by Dr. Tatsuaki Okamoto, NTT
Fellow.

Photo 5. S pecial session by Dr. Pratish Datta, NTT Secure
Platform Laboratories.

exposing them to danger.
Following her lecture, Dr. Tatsuaki Okamoto, NTT
Fellow, gave a talk entitled “Diversity and COE—
Cryptologic Research in NTT: Talk with Dr. Datta, aVisually Handicapped Researcher” (Photo 4). Dr.
Okamoto explained that NTT Secure Platform Laboratories is a center of excellence (COE) with a global
reputation in the field of cryptography and that diversity is the key to this excellence. Since 1988, whenthe Laboratories first invited researchers from outside
NTT, each year they have accepted many researchers
from both inside and outside Japan. If we count only
those since 2015, 55 researchers from more than 20
countries have resided there.
Then, Dr. Pratish Datta went to the podium andspoke of his first encounter with Dr. Okamoto, theresearch environment of NTT, and his life at theLaboratories (Photo 5). He explained that he is able
to conduct his research in an excellent environment
with top-notch cryptography researchers from aroundthe world and that they voluntarily help each otherout. Dr. Datta, who has visual and hearing disabilities, added that he has been receiving sufficient support for his research activities and that he will continue his research at NTT to develop theoretically andpractically refined cryptography.
The next day, under the title of “Leading in the Ageof Digital Disruption,” Marc Alba, Chief Disruption
Officer at everis, elaborated on his perception that allindustries in the world today, from banking, medicalcare, automobiles, to retailing, are in the process of
reinventing themselves, and that what is needed is

Photo 6. S pecial session by Marc Alba, Chief Disruption
Officer, everis.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019disruptive innovation for solving core issues rather
than improvement of existing technologies (Photo 6).
Based on this perception, he proposed that, as part ofits disruptive innovation initiatives, the NTT Group
create new businesses by returning the ownership of
personal data—which have traditionally been entrusted to service providers—to individuals and thenusing such personal data.
The lectures were an introduction to the activities
of NTT R&D and the NTT Group, and were warmly
received by the audience.

76

Information

Photo 7. Media and UI.

Photo 8. AI.

3. Exhibition of research results

ing of sports on a wide screen using synthesized andeconomized four 4K video streams (Photo 7).

At the forum venue, 124 of the latest research
results were on display under six categories: media
and user interface (UI), artificial intelligence (AI),IoT, network, security, and basic research. Thus, theexhibition covered a wide range of research findings,
from technologies being developed by the NTT
Group to outcomes resulting from collaboration withpartner companies, and from basic research to alreadycommercialized technologies.
To enable visitors to see the results of R&D more
effectively, we created for the first time an exhibitionsite dedicated to introducing smart world-related
activities that cut across different exhibit themes, inaddition to the exhibition sites dedicated to specifictopics. At these sites, visitors were able to see welldesigned demonstrations and exhibits.
3.1 Media and UI(1)	Media technology for providing new user experience
The ultra-realistic communication technology
Kirari! was introduced in a lineup of four presentations, each adapted to a specific usage, along withassociated elemental technologies. The first and main
one was a theater-type exhibit aimed at creating awhole new level of excitement. The second was“Kirari! for Arena,” which transmits images of an
entire competition space and enables the audience ata remote site to view the space from all directions inreal time. The third was a highly realistic reception
window. The fourth presentation featured broadcastNTT Technical Review

Vol. 17 No. 2 Feb. 2019(2) UI technologies targeted at 202X
An introduction was given of technology enabling
low-cost collection and updating of barrier-free information needed by elderly or handicapped people forwalking, as well as walking guide technology, with aview to achieving diversity navigation, which makes
daily-life activities safe and convenient for those withmobility issues. In addition, technology that enables
robots to present information more naturally was on
display.
3.2 AI(1) AI that supports people
The following technologies were introduced: Totto,
the interviewer robot, which accurately recognizes
what someone is saying and engages in a natural dialogue; RexSense®, an engine that recognizes nonverbal information (a speaker’s attributes, feelings,
and intentions) from speech; and technology that
understands a conversation between a customer andan operator, infers the customer’s situation from their
responses, and extracts predefined key issues such as
the subject matter and the customer’s reason for calling, in real time (Photo 8).
(2) AI that supports society
Also on display were AI technologies able to perform optimal control based on the numbers and flowsof people in and out of a site. They included technology that uses AI to control air conditioning of a large
77

Information

Photo 9. IoT.

Photo 10. Network.

commercial facility with an unspecified number ofpeople coming in and out in order to reduce air conditioning costs while keeping the facility comfortable, and technology that uses a learning simulator to
predict flows of people and thereby generate a guidance plan for various places such as an event site, atourist spot, or a transportation facility.
(3) Fundamental technologies
Also presented were fundamental learning technology to accelerate deep learning, technology that converts and compresses a large volume of spatial data
into a group of images, and technology able to deliver
AIs to appropriate points within a network and makethem operate in coordination.

(3) Analytics and prediction
The latest research results that combine big data
analysis and deep reinforcement learning were introduced (Photo 9). They included a public security
solution that quickly detects and deals with an incident through analysis at a micro-datacenter located in
the vicinity of the monitored area and technology thatpredicts where and when incidents that require an
emergency response are likely to occur and optimally
allocates rescue teams.

3.3 IoT(1) Sense, connect and drive
Technologies that digitize and transmit data that areread out from various objects were introduced, forexample, technology for the sensing fabric, hitoe,
which is being employed in a wide range of fields
including healthcare; and technology for reliably collecting data from sensors in places where reception ofmobile radio waves is weak.
(2) Data and software logistics
On display were IoT data exchange technology that
achieves high-speed data exchange between a variety
of machines and applications in the fields of manufacturing, automobiles, and ships, and edge computing technology that achieves real-time distributed
processing for handling a large volume of data withlow latency.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

3.4 Network(1) Flexible and high-speed networks
A wide range of network technologies for increasing network capacity or for optimizing network functions to suit different service requirements wereshowcased (Photo 10). These included wirelesstransmission technology for the post-5G (fifth-generation wireless communication networks) era andnetwork slice technology for generating a network
suitable for different needs.
(2)	Networks that rapidly restore and provide services
The following technologies were introduced: operation technology that maximizes the scope of zerotouch operations and planned maintenance by determining how to handle various operational events suchas service applications from customers and faults,
based on an SLA (service level agreement), and technology that uses Network-AI to assist operators in
different phases, from monitoring, analysis, and control to recovery, with the aim that in the future it will
78

Information

Photo 11. Security.

Photo 12. Basic research.

be possible to handle faults that occur in ICT systems
automatically.

transparent battery that will enable researchers to create a device that is virtually invisible; child language
development analysis technology, which identifies
the characteristics of late talkers; middleware andapplications for LASOLV, a quantum neural networkthat solves challenging problems at an extraordinary
speed; and coating-type deterioration sensing technology, which detects anomalies in structures simply
by coating them.

(3) Network/IT (information technology) solutions
Solutions that utilize network functions and technologies that can be employed immediately for service development were exhibited. They included the
API (application programming interface) orchestrator, which supports the public security solution in Las
Vegas, and solutions that were the result of collaborations with partner companies.
3.5 Security
Unknown cyber-attack detection technology andpersonal information anonymization technology
were exhibited (Photo 11). These consist of leading
security technologies—some defensive and someoffensive—that strengthen measures against cyber­attacks and will lead to the early realization of a datacentric society.
3.6 Basic research
Results of leading-edge basic research aimed atcreating new principles or concepts in the coming
years were introduced (Photo 12). They included a

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

4. Conclusion of the forum
To enable many people, including those fromabroad, to come to the forum, we moved the forum
dates from spring, when it was formerly held, toautumn. Consequently, we were able to welcome
more than 15,000 visitors. We believe that what lies
behind this large number of visitors, many from overseas, is their high expectation for NTT’s R&D. In
fact, visitors to the exhibit sites and those who took
the questionnaire survey clearly expressed the expectations they had for our R&D. To live up to their highhopes, we will redouble our efforts in all areas frombasic research to the development and deployment of
new technologies.

79Information

Authors (from left): Atsuyuki Muramoto, Manager, R&D
Planning, NTT Research and Development Planning
Department; Hideo Kato, Manager, R&D Planning, NTT
Research and Development Planning Department; Takashi
Ikebe, Manager, R&D Vision Group, NTT Research andDevelopment Planning Department; Norio Sakaida,
Manager, Research Planning Department, NTT Science
and Core Technology Laboratory Group; Masaki Hisada,Manager, R&D Management, Planning Department, NTT
Service Innovation Laboratory Group; and Kentaro Hotta,
Manager, Planning Department, NTT Information Network
Laboratory GroupNTT Technical Review

Vol. 17 No. 2 Feb. 201980

Short Reports

Controlling Nuclear Magnetic
Resonance with a MicromechanicalOscillator—A Novel Technique to
Individually Manipulate Nuclear
Spins in Integrated Devices
1. Introduction
NTT and the National Institute of Advanced Industrial Science and Technology (AIST) have developed
a novel device technology to control solid-state
nuclear magnetic resonance (NMR) with a micromechanical oscillator. Quantum memory has received
much attention in recent quantum technology
research for applications to quantum computers,
quantum communications, and quantum sensors.
Quantum computers are expected to perform ultrafast
computation, while quantum communications canprovide absolutely secure communications protocols,and quantum sensors promise to drastically improvesensor performance.
Quantum memory’s ability to store quantum superposition states for a long period of time could improve
the performance of these quantum technologies. Oneof the most important candidates for quantum memory is nuclear spin because of its extremely long spin
lifetime. NTT and AIST experimentally demonstrated that the NMR frequency in a mechanical oscillator
can be controlled by strain induced by its mechanicalmotion. This method enables us to control the quantum state of a nuclear spin in a desired mechanicalelement in an integrated array and will play an essential role in the development of solid-state-based quantum memory.
This demonstration was made possible through acollaboration between NTT, which performed device
fabrication and measurements, and AIST, which performed data analysis based on precise theoreticalmodeling. The results were published in Nature Communications on August 28, 2018 [1]. This work was
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

partly supported by a Grant-in-Aid for Scientific
Research on Innovative Areas (KAKENHI) from theJapan Society for the Promotion of Science, specifically by the research project of Science of Hybrid
Quantum Systems.
2. Research background
Nuclear spin is the rotation of an atomic nucleus,
and it shows a precessional motion under a magnetic
field. Nuclear spin can sustain precessional motion
for a long period of time and is expected to be used invarious applications such as quantum memory tostore a quantum superposition state and high-performance magnetic field sensors. NMR is generally used
to manipulate the nuclear spin state, where an electromagnetic wave at the NMR frequency is applied
under a magnetic field to modify the precessionalmotion. However, it is hard to use standard NMR
techniques for integrated devices because the magnetic field and electromagnetic wave are simultaneously applied to all the devices. Therefore, a novel
method to manipulate nuclear spins in individual
devices is in high demand.
3. Research results
The NMR frequency is highly sensitive to the strain
applied in a solid, and this property was used to
locally manipulate nuclear spins in this experiment.
Artificially generating strain at a desired position in amaterial allows the control of NMR in individual
integrated devices. A micromechanical oscillator fabricated using state-of-the-art nanofabrication technology
81Short Reports

Va

[011]

Alternate voltage
for actuation

Detectionof oscillating
motion

10 µm
[011]

Vd

(a)

Periodic strain is induced in this area.

Nuclear spins

Conductive layer

Current

StrainVibration
Frequency of mechanical vibration:
fM = 1.716 MHz

Antenna ofelectromagnetic
wave
Spin state is detected through device resistance.

(b)

(c)

Fig. 1. Mechanical oscillator used in this study. (a) Electron microscope false-color image of a micromechanical oscillator
used to apply strain on nuclear spins. The mechanical flexural motion is induced by applying an alternate voltage on
the surface electrode (shown in yellow). (b) Schematic drawing showing the generation of strain. The flexural motionof a 50-µm-long beam induces periodic strain on the clamping area. (c) Magnified schematic image of strain area ofthe oscillator. An antenna radiating electromagnetic waves generates an oscillating magnetic field and induces
nuclear magnetic resonance. The change in the nuclear spin state is detected as the change in resistance.

was used to locally generate large strain, and the
NMR frequency was controlled with oscillatingstrain using the resonator’s mechanical motion. As aresult, we observed not only the modulation of the
NMR frequency but also a novel kind of NMR called
sideband resonance, which were induced by themixed effect of electromagnetic waves and oscillating strain.
4. Experimental details
The mechanical oscillator (Fig. 1(a)) has the shapeof a doubly clamped beam and was fabricated byprocessing GaAs (gallium arsenide), which is apiezoelectric semiconductor. The mechanical oscillation can be electrically induced through the piezoelectric effect. The motion generates periodic strain at
the clamping point (Fig. 1(b)) and modifies the
dynamics of nuclear spins in the strained region. In
the experiments, we observed not only a shift in the
NTT Technical Review

Vol. 17 No. 2 Feb. 2019

NMR frequency but also sideband resonances, where
the resonance was also confirmed at two shifted frequencies: blue- and red-detuned from the original
NMR frequency. The theoretical calculation factored
in the effect of oscillating strain on nuclear spin
dynamics and showed good agreement with theexperimental results, proving that the observed phenomenon can be well explained by the effect of
mechanical oscillation.
5. Outlook
The mechanical oscillators are fabricated using
semiconductor nanofabrication technology and canbe integrated on a tiny semiconductor chip. We canutilize this scheme as a novel platform to integratequantum memories and sensors by demonstrating
nuclear spin control in individual arrayed devices.

82

Short Reports

Reference[1]

Y. Okazaki, I. Mahboob, K. Onomitsu, S. Sasaki, S. Nakamura, N.
Kaneko, and H. Yamaguchi, “Dynamical Coupling Between a Nuclear Spin Ensemble and Electromechanical Phonons,” Nat. Commun.,Vol. 9, 2993, 2018.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

For Inquiries
Public Relations,
NTT Science and Core Technology Group
http://www.ntt.co.jp/news2018/1808e/180828b.html

83External Awards
38th CHEMINAS Poster Award
Winner: Hiroki Miyazako, Tetsuhiko Teshima, and Yuko Ueno, NTT
Basic Research Laboratories
Date: November 1, 2018
Organization: Society for Chemistry and Micro-Nano Systems(CHEMINAS)

APMC 2018 Prize
Winner: Hiroshi Hamada, Takuya Tsutsumi, Hiroki Sugiyama, andHideyuki Nosaka, NTT Device Technology Laboratories
Date: November 9, 2018
Organization: 2018 Asia-Pacific Microwave Conference (APMC
2018)

For “Direction of Topological Defects in Cell Population by
Computer-aided Design of Geometrical Boundaries.”
Published as: H. Miyazako, T. Teshima, and Y. Ueno, “Direction ofTopological Defects in Cell Population by Computer-aided Design ofGeometrical Boundaries,” 38th CHEMINAS, 1P28, Sapporo, Japan,
Oct./Nov. 2018.

For “High-output-power and Reverse-isolation G-band Power
Amplifier Module Based on 80-nm InP HEMT Technology.”
Published as: H. Hamada, T. Tsutsumi, H. Sugiyama, and H. Nosaka, “High-output-power and Reverse-isolation G-band Power Amplifier Module Based on 80-nm InP HEMT Technology,” Proc. of
APMC 2018, Kyoto, Japan, Nov. 2018.

Papers Published in Technical Journals andConference Proceedings
Merlin-Arthur with Efficient Quantum Merlin and Quantum
Supremacy for the Second Level of the Fourier Hierarchy
T. Morimae, Y. Takeuchi, and H. Nishimura
Quantum, Vol. 2, p. 106, November 2018.
We introduce a simple sub-universal quantum computing model,
which we call the Hadamard-classical circuit with one-qubit (HC1Q)
model. It consists of a classical reversible circuit sandwiched by twolayers of Hadamard gates, and therefore it is in the second level of theFourier hierarchy. We show that output probability distributions of
the HC1Q model cannot be classically efficiently sampled within amultiplicative error unless the polynomial-time hierarchy collapses
to the second level. The proof technique is different from those used
for previous sub-universal models such as IQP, Boson Sampling, and
DQC1, and therefore the technique itself might be useful for finding
other sub-universal models that are hard to classically simulate. We
also study the classical verification of quantum computing in thesecond level of the Fourier hierarchy. To this end, we define a promise
problem, which we call the probability distribution distinguishability
with maximum norm (PDD-Max). It is a promise problem to decide
whether output probability distributions of two quantum circuits are
far apart or close. We show that PDD-Max is BQP-complete, but if
the two circuits are restricted to some types in the second level of theFourier hierarchy such as the HC1Q model or the IQP model, PDDMax has a Merlin-Arthur system with quantum polynomial-time
Merlin and classical probabilistic polynomial-time Arthur.

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Development and Evaluation of an Applause and Handclapping Sound Feedback System to Improve Realistic
Feeling on Live Viewing
A. Fujimori, K. Kawahara, Y. Kamamoto, T. G. Sato, M. Nishikawa, A. Omoto, and T. Moriya
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. J101-A, No. 12, pp. 273–282,
December 2018 (in Japanese).
Considering the increasing popularity of public viewing, improvement of immersive content is becoming an important issue. This
paper presents APRICOT: APplause for Realistic-Immersive COntents Transmission system as one of the solutions for enhancing thequality of live-viewing services. APRICOT detects the sounds ofapplause and hand-clapping at a remote site, transmits the information as meta-data, and represents the data at the host site. Since handclapping is the most common response for an audience and is simple
to replay, the system can transmit the excitement in real time with low
delay. Experiments indicate that feedback of the applause and handclapping enhances the immersive sound, especially at the host site.

Impossibility of Blind Quantum Sampling for Classical Client
T. Morimae, H. Nishimura, Y. Takeuchi, and S. TaniarXiv:1812.03703 [quant-ph], December 2018.
Blind quantum computing enables a client, who can only generateor measure single-qubit states, to delegate quantum computing to aremote quantum server in such a way that the input, output, and84

Papers Published in Technical Journals and Conference Proceedings

program are hidden from the server. It is an open problem whether acompletely classical client can delegate quantum computing blindly.
In this paper, we show that if a completely classical client canblindly delegate sampling of sub-universal models, such as the DQC1
model and the IQP model, then the polynomial-time hierarchy collapses to the third level. Our delegation protocol is the one where theclient first sends a polynomial-length bit string to the server, and thenthe server returns a single bit to the client. Generalizing the no-go
result to more general setups is an open problem.

synchronously moved to simulate the sensation of real walking.
These motion profiles do not completely follow the measured data ofreal walking, but each gain of magnitude of the modalities is adjusted
on the basis of the subjective intensity of motion impression, mainly
due to the lack of the sensory suppression from the motor commandof the participant. The demonstration of our exhibition booth presents
a virtual trip to tourist sites in Toronto and Niagara Falls, Canada. A
short-time experience of walking around the area is relived/shared by
the attendee.

A Supervised Learning Approach to Granger Causality
Inference
Y. Chikahara and A. Fujino
IPSJ Transactions on Mathematical Modeling and Its Applications, Vol. 11, No. 3, pp. 58–73, December 2018 (in Japanese).
Granger causality is one of the definitions of temporal causality
between variables, and inferring Granger causality is an importanttask in time series analysis. Traditional methods use regression models for this task. Since the inference accuracies of these methods
depend largely on whether or not we select an appropriate regression
model for each time series data element. However, it is not easy
because such selection of regression models requires a deep understanding of data analysis. This paper proposes a supervised learning
framework that utilizes a classifier instead of regression models. Our
proposed method employs a feature representation that utilizes the
distance between the conditional distributions given past variablevalues. We experimentally show that the feature representation gives
sufficiently different feature vectors for time series with different
Granger causality.

Leg-Jack: Generation of the Sensation of Walking by Electrical and Kinesthetic Stimuli to the Lower Limbs
H. Kaneko, T. Amemiya, V. Yem, Y. Ikei, K. Hirota, and M.
KitazakiProc. of SIGGRAPH ASIA 2018, Tokyo, Japan, December 2018.
We developed a neurosensory and kinesthetic stimulation systemthat generated a walking sensation for a seated user. An electricalstimulus was applied to the Achilles’ and tibialis anterior tendons
with a kinesthetic stimulus generated by a lower limb device driven
synchronously with an egocentric visual scene during virtual walking. The system works as part of an experience replication schemethat aims to receive the physical activity of another (person, entity,etc.). As a common bodily activity of humans, walking motion was
focused on. The evaluation experiment has shown that walking sensation was increased by each stimulation at a 1% significance level. In
this demonstration, the user on a chair can feel as if he/she is walking
in a haunted house. The user can move the upper body freely to lookaround with a virtual flashlight; however, the lower body is possessedby the other entity. The user walks into the house despite his/her
intention. The work gives the user a realistic experience which was
not sufficiently generated with only a movie and sounds.

FiveStar VR: Shareable Travel Experience through Multisensory Stimulation to the Whole Body
K. Shimizu, G. Sueta, K. Yamaoka, K. Sawamura, Y. Suzuki, K.
Yoshida, V. Yem, Y. Ikei, T. Amemiya, M. Sato, K. Hirota, and M.
KitazakiProc. of the 11th ACM SIGGRAPH Conference and Exhibition on
Computer Graphics and Interactive Techniques in Asia (SIGGRAPH
ASIA 2018), Tokyo, Japan, December 2018.
We have developed a multisensory virtual reality system, FiveStar
VR (five senses theater for VR), that enables participants to relive or
share each other’s behavior through well-designed simultaneous
stimulation to multiple modalities. FiveStar VR consists of somatosensory displays in addition to the conventional audio-visual VR
setup. In the FiveStar VR, the body parts of the participant are forced
to move, which is synchronized with those of an avatar in the VR
space, resulting in inducing a strong perception of presence at thepast walking behavior of someone else. By taking advantage of thecyclic nature of walking, the arms, the lower limbs, and the body are

NTT Technical Review

Vol. 17 No. 2 Feb. 2019

Channel Code Using Constrained-random-number Generator Revisited
Jun Muramatsu and S. Miyake
IEEE Transactions on Information Theory, Vol. 65, No. 1, pp.
500–510, January 2019.
A construction of a channel code by using a source code withdecoder side information is introduced. The encoder and decoder pairof any source code can be used for the construction. Constrainedrandom-number generators, which generate random numbers satisfying a condition specified by a function and its value, are used toconstruct stochastic encoders and decoders. The result suggests that
we can divide the channel coding problem into the problems of channel encoding and source decoding with side information.

85