No.
4
2023
500

FEATURE

Demonstration

Aiming for a Worldof Mutual
Understanding with AI
Miss MURAYAMA Kirari “Plays in English”
with NICT’s AI SimultaneousInterpretation and VoiceTra®

th

Anniversary Issue

Aiming for a World of Mutual Understanding with AI

3

4

500

This image isunavaibable due tothe contract.

Demonstration

Miss MURAYAMA Kirari “Plays in English”
with NICT’s AI Simultaneous Interpretation
and VoiceTra®

Aiming for a World of Mutual
Understanding with AI
Demonstration

1

Miss MURAYAMA Kirari “Plays in English”
with NICT’s AI Simultaneous Interpretation
and VoiceTra®
MURAYAMA Kirari × UCHIMOTO Kiyotaka

4

Development of Speech Communication Technologies

Celebrating the 500th issue of NICT NEWS, we invited
Miss MURAYAMA Kirari (a 7th grader), a TV personality
who appeared regularly on the NHK-E program "Let’s
Play in English with Orton" and is fluent in English, to ourheadquarters to experience the latest speech translation
technologies that are currently being developed and“VoiceTra®,” a multilingual speech translation app that NICT
has been developing over the years. The demonstrations of
the cutting-edge translation technologies turned out to bea succession of surprises for Kirari-san.
Commentary by Dr. UCHIMOTO Kiyotaka, Director Generalof Universal Communication Research Institute.

KAWAI Hisashi
6

Automatic Simultaneous Interpretation Technology
Technology that automatically interprets spoken words without delay

Born April 8, 2010.
Her hobbies are participating in triathlons and playing theocarina. She appeared regularly on the NHK-E program "Let’ s
Play in English with Orton" for four years (from FY2017 toFY2020) and is fond of English. She is now in her 7th grade (1stgrade of junior high school) and is also highly interested in
the Arabic language.

Director General, Universal Communication Research Institute
He joined the Communications Research Laboratory of the
Ministry of Posts and Telecommunications (now NICT) in 1996.
He has been engaged in research and development of natural
language processing and the social implementation of speechtranslation technologies to return the achievements ofresearch to society. Ph. D. (Informatics).

UTIYAMA Masao
8

Data-driven Intelligent Communication Technology
OHTAKE Kiyonori

10

Smart Data Analytics Technology

Predictive analysis and behavior support through sensing data collaboration

VoiceTra®

ZETTSU Koji

12

History of Changes up to NICT NEWS 500th Issue
Press Office, Public Relations Department

13

NICT's Challenger File 25 OKAMOTO Takuma
Multiple Sound Spot Synthesis System: a Next-generation
Technology Capable of Simultaneously Synthesizing Different
Sounds in Different Zones

14

AWARD

YouTube videos introducing our research, "NICT Station," released!
In June 2023, YouTube videos "NICT Station" which aims to have people understand research outcomes of NICT was released.
In the videos, researchers and NICT's official character "N" provide explanations on the following
nine themes to a reporter who visited NICT. They explain the content and social
impact of individual research themes in a joyful and easy-to-understand manner.
The narrator is KAMISHIRAISHI Mone, a talented young actress, who also narrated
in a video titled "A Future with N." Please enjoy the videos that will make you feel
impatient for the future.
https://www2.nict.go.jp/publicity/nict_station/(in Japanese)

UCHIMOTO Welcome to our institute, Kirari-san. The Universal Communication Research Institute promotes research and development of technologies to realize universal
communication where everyone can understand each other with ease. It is research tosupport all kinds of human activities using AI.
Automatic translation technology, which I
want to introduce to you now, is one form ofsuch technology, and it allows anyone to easily overcome the language barrier when communicating with someone who speaks a different language. First, I would like you to try
out VoiceTra®, a multilingual speech translation app that we developed. After that, we
would like you to experience the AI simultaneous interpretation technology that can beused on occasions like international conferences.

translation app that can be used free of charge
simply by downloading the app on your smartphone. It supports both iOS and Android OS.
Open the app, tap the microphone button on
the screen, and speak in Japanese—yourspeech will be automatically translated into
English and you will be able to hear the audio
output. Please say something.
KIRARI "Konnichiwa, watashi ha MURAYAMA Kirari desu."
VoiceTra“Hello! I am Kirari Murayama.”

KIRARI Wow! We can hear the English
translation!
UCHIMOTO How about a longer sentence?

plicated dialogue can be translated instantly.
Both the translated English and the back translation into Japanese are shown on the screen.
Using this, I can probably have a carefree conversation with people from overseas.
UCHIMOTO Now, let’s try a conversationthat is closer to a real-life situation using
VoiceTra®. You have just arrived at the airport in Los Angeles. You want to ask someone
for a sightseeing recommendation. Just at theright moment, a kind-looking person passes
by. It’s Ananda.
KIRARI "Sumimasen. Kanko ni ikitainodesuga, Los Angeles no yumeina bashowo oshiete itadakemasenka."

KIRARI "Asukara atsuku narisounanode,necchuushou niha kiwo tsukete kudasaine."

VoiceTra “Excuse me, I’d like to go sightseeing. Could you tell me some famous placesin Los Angeles?”KIRARI Thank you. I’m very excited.

VoiceTra “It will be hot from tomorrow, so
please be careful of heatstroke.”

ANANDA "Nice to meet you. I recommend
Griffith Observatory. Do you like stars?"

UCHIMOTO VoiceTra® is a multilingual

KIRARI It’s impressive that even quite com-

VoiceTra “Konnichiwa, Griffith Temmond-

NICT NEWS 2023 No.4

1

Aiming for a World of Mutual Understanding with AI

Demonstration

Miss MURAYAMA Kirari “Plays in English”
with NICT’s AI Simultaneous Interpretation
and VoiceTra®

ai e ikukotowo osusumeshimasu. Hoshi wa
sukidesuka?”
KIRARI "Hai, namae ni hoshi to iu mojiga
haitteirunode daisuki desu."
VoiceTra “Yes, I love stars because myname has the word star in it.”UCHIMOTO You can carry a conversationin such way.
KIRARI I’m surprised that the app speaks
both English and Japanese so fluently. It’s really good that the app accurately translates
what I want to say and conveys it to the person
I’m talking to. It’s so nice that you can talkwith people around the world normally using
Japanese. I think having conversations withpeople in other countries will help to expand
your knowledge and views.
AI Simultaneous Translation -Presentation-

UCHIMOTO Tapping the translation button
like VoiceTra® may hinder smooth communication when it comes to high-paced conversations or long dialogues, since there will be atime lag. Thus, we are currently developing atechnology called "AI simultaneous interpre-

tation." Now, please try this. It can be used foroccasions like international conferenceswhere people from different countries speak
in different languages. This would allow theaudience to hear others speak in their own languages.
Let’s ask Ananda, a native English speaker,
to give a speech in English. Her speech will be
instantly translated into Japanese and you canhear the audio output as well.
This system is being developed with the assumption that it will be used at EXPO 2025,
Osaka, Kansai, Japan, under the theme of"People’s Living Lab."

person is saying as if you are actually listening
to their speech. Many international conferences and keynote speeches by speakers fromoverseas are scheduled to be held at Expo
2025 in Osaka, so we hope this will help eliminate the language barriers in such occasions. I
think this is something that has never been
done in the history of the Expo.
Another goal of this translation technology
is to keep the users unaware of the advanced
technology used. We want to commoditize thetechnology to such an extent that people willuse it without even thinking about it, just like
we do with a pencil or an eraser.

KIRARI The speech is being constantly
translated just like a human simultaneous interpreter is speaking. How does it judge where
to cut within a speech?

KIRARI Technology as accessible as a pencil! Awesome!

UCHIMOTO AI automatically identifies thesegments in a sentence and translates inchunks that contain meaning. This wasachieved by training the AI with experiencesof human simultaneous interpreters.
KIRARI So that’s why it’s translated intosuch beautiful and proper Japanese!
UCHIMOTO Since the translation is generated constantly, you can understand what theThis image isunavaibable due tothe contract.

Miss Kirari experiences remote meeting using
NICT's AI simultaneous interpretation technology

2

NICT NEWS 2023 No.4

UCHIMOTO There are so many languages
in the world, but Japanese and English is actually one of the most difficult combinations fortranslation. For instance, the word order iscompletely different between the two languages. In English, the verb comes after the
subject, but in Japanese, the verb comes at theend of the sentence. You cannot understand aJapanese sentence until you hear it until theend. Therefore, if Japanese-English translation—which is very difficult—becomes possible, the technology can be also applied to thetranslation of other languages.
Japanese is merely one of the many languages in the world, so it tends to be less prioritized in services overseas. However, as Japanese citizens, we would like to set Japaneseas the basis. NICT has the advantage of possessing a vast amount of translation data thatthe institute has been accumulating for many
years, and most of it is in Japanese.
AI Simultaneous Interpretation -Teleconference-

UCHIMOTO Next, let’s try a teleconferencewith someone supposedly far away in a foreign country using AI simultaneous interpretation technology. It is useful not only forcompanies that communicate frequently withpeople abroad but also for private communication with friends overseas who speak different languages, so it can broaden your horizons.
Now, let’s suppose you are visiting the U.S.
in the near future and have a chat with Ananda, who lives in Los Angeles (but in reality issitting in the next room).
KIRARI "America de zehi itta houga yoi
tokorowa arimasuka?"
Translated Speech in English “Are there
any places you’d like me to visit in America?”
ANANDA “Well, America is so big, so
there are many, but you should go to the Grand
Canyon in Arizona and the rainforest in Washington State.”
Translated Speech in Japanese “So ne,
America wa totemo hiroi kara takusan arukedo Arizona-shuno Grand Canyon to Washington-shuno nettaiurin niwa ikubekidayo.”
KIRARI "Grand Canyon ha kiitakotoga arimasu. Zehi ittemitaidesu."
Translated Speech in English “I’veheard of the Grand Canyon. I'd love to go
there.”
ANANDA “Yes, it’s a very beautiful landscape, wide open space. It’s not a crowded
place like Tokyo. All you can see are stars andnature.”Translated Speech in Japanese“Eh,asokowa totemo kireidesushi, hiroku aita kukan desu. Tokyo no yona konzatsushita basho
denaku, mirukotoga dekirunowa hoshi to shizen dakedesu.”
KIRARI "Watshiwa hoshi mo shizen mo
daisuki nanode ikunoga tanoshimidesu."
Translated Speech in English“I likestars and nature, so I’m looking forward togoing there.”
UCHIMOTO Conversation with someone
who speaks a different language will go on
like this. How was it?

This image isunavaibable due tothe contract.

Miss Kirari experiences multilingual communication using VoiceTra®

KIRARI There are some words that I know,
but there are also parts I that couldn’t makeout, because native speakers talk fast and fluently, so it is really helpful that those parts aretranslated.

Future of Translation Technology

UCHIMOTO AI translation technology has
already come this far. We would like to further
increase the accuracy of translation and makeit available for ordinary use. Kirari-san, how
did you like communicating in a foreign language using our translation technology today?
KIRARI Usually, I don’t have the courage totalk with people from overseas even if I wanted to. I’m often worried that I won’t be able toclearly express what I want to, so I hesitate tospeak. But if this kind of translation technology is available, I am sure that I can convey
anything I want to. If I can talk with local people in English, I might be able to gain knowledge that I can’t get from textbooks, so I think
this is great.
UCHIMOTO Certainly. You can ask about
various things you do not know, which you
can understand in your native language; thistechnology may also be used to expand yourknowledge. It also allows you to communicate
in languages other than English too.
KIRARI I would like to have conversations
using the translation app because I don’t haveany knowledge of or experience in foreign
languages other than English.

to let the machine do the work for certain cases.
The translation technology that we are developing is learning from so-called “accumulated experience,” which is a collection ofmany case examples of parallel translation
data. Whether it is English or another language, we collect a lot of experience on how
sentences are translated.
The goal is to achieve technology that cantell you what you really want to know. Also, inbusiness settings, there may be cases where
negotiation is necessary. Therefore, we would
like to develop a translation technology that
can also cover social and cultural backgrounds
so that people can understand each other in aneven more natural manner.
KIRARI Yes, it would be even better if thetranslation could read the room and nuances ofthe situation.
UCHIMOTO In order to make AI translation system more sophisticated, a vast amountof parallel translation data is necessary. We
would like to collect as much of these data as
possible while being careful to prevent important intellectual properties from leaking tomarkets abroad, and develop technologies
original to Japan.
KIRARI I’m looking forward to the future oftranslation technology. It’s so exciting to hearthat such advanced technology will be commonly available at EXPO 2025 in Osaka.
Thank you very much for everything today.
Oh, my manager has downloaded VoiceTra®
on her smartphone, so I will continue using it!

UCHIMOTO It takes a huge amount of timeto learn a new language, so I think it is better
NICT NEWS 2023 No.4

3

Aiming for a World of Mutual Understanding with AI

Development of Speech Communication Technologies

n order to achieve the Global Communication Plan 2025 (GCP2025), which
was announced by the Ministry of Internal Affairs and Communications in March 2020, theAdvanced Speech Technology Laboratory has
been working on three challenges, 1) R&D of
speech recognition, language identification,and speaker recognition technologies forautomatically converting speech of lecturesand business/international conferences into
texts in the primary languages, such as Japanese, English, and Chinese, with practical precision, 2) R&D of technology for synthesizing
human-level speech to smoothly convey thetranslation results, etc. and technology forcontrolling voice quality, such as speech rate,
while curbing deterioration in naturalness tothe extent possible, and 3) expansion of the
supported languages for speech recognition
and text-to-speech regarding daily conversations.

I

KAWAI Hisashi
Director of Advanced Speech Technology Laboratory, Advanced Speech
Translation Research and Development Promotion Center (ASTERC),
Universal Communication ResearchInstitute
He received his Ph.D. in Engineering in
1989. He joined the Kokusai Denshin
Denwa Co. Ltd. (currently KDDI Corporation) in April 1989. He has been engaged
in research on speech synthesis andspeech recognition. From 2000 to 2004,
he worked for the Advanced Telecommunications Research Institute International(ATR). He joined NICT in October 2014.

■ Speech Recognition of Lectures andMeetings
In the process of enhancing accuracy inspeech recognition, the creation of large-scalespeech corpora (data accumulating a large
amount of speech and text of the content thereof) for training speech recognition models while
simultaneously improving modeling methods
is critical. Speech in lectures or meetings areless clear as speakers' levels of consciousness
for articulation are lower than in the case of
carefully speaking sentence by sentence tosmartphones using speech translation applications, and the distribution of acoustic features isalso apt to vary. When trying to recognize suchspeech using an acoustic model trained on clear
speech, the speech feature quantity deviatesfrom the identification boundary and the accuracy in speech recognition deteriorates significantly. In order to deal with this problem, wetrained acoustic models by adding a corpus ofless clear speech at a level representing around
30% of the total to the existing corpus of clear
speech (2,000 hours for the Japanese, English,
Chinese, and Korean language, and 1,000 hours

4

NICT NEWS 2023 No.4

for other languages) to make the identification
boundary adapt to such unclear speech. Materials for the acoustic corpus were mainly collected from mock lectures and meetings in order toavoid the issues of confidentiality, protection of
personal information, copyrights, etc., but theyalso include real speech from open lectures andmeetings within NICT, and the acoustic corpus
is being used for training speech recognition
models and evaluating the accuracy.
By FY2022, we created a speech corpus
of around 3,000 hours in total mainly in Japanese, English, Chinese, and Vietnamese andachieved recognition with practical precisionin Japanese and recognition with accuracy atan almost practical level in English. Generally,
speech recognition accuracy is measured by theword error rate, but the rates fluctuate significantly depending on the content of the speech
data for testing. Accordingly, the accuracy is
described using the terms shown in Table 1.
Furthermore, we improved the algorithms andacoustic models and halved the time lag fromthe input of speech into the speech recognitionengine to the output of the results, which had
been 4.1 seconds. While making these steadyefforts for improvement, we are working on
studies for practical realization of end-to-end
speech recognition in which a string of characters is directly output by inputting speech signals into a neural network.
In language identification, when acoustic
features differ between speech data to be used
for identification model training and real speech
data which is the target of identification, theproblem of domain dependency arises, and theidentification accuracy deteriorates significantly. It is thus difficult to secure identification accuracy at a practical level mainly for languages
for which real speech data for training are hard
to collect. As possible causes, over-fitting tocharacteristics of surrounding noise when recording and frequency response characteristics
of microphones were suspected; therefore, we
devised the Transducer-based language embedding method which allows modeling of linguistic features (features of a chain of phonemes,
the minimum units of spoken language), in

Figure 1T ransducer-based language embedding to identify languages in consideration of both acoustic
features and linguistic features

Table 1

Classification of speech recognition accuracy

S

Human level

The results of speech recognition can be read and understood
without any problem.

A

Practical level

The results of speech recognition contain minor errors but canbe read and understood sufficiently.

B

Almostpractical level

The results of speech recognition contain errors but can be readand understood to some extent.

C

Experimentallevel

The results of speech recognition contain many errors and are
difficult to understand.

D

Prototype level

The results of speech recognition contain many errors and are
extremely difficult to understand.

addition to acoustic features (Figure 1). When
we evaluated the language identification accuracy by applying this method to training datathat recorded a mock conversation and real data
for testing, the identification error rate for a
two-second speech almost halved from 17.7%.

■Human Voice Text-to-Speech
As for text-to-speech, we are also working
on the development of speech corpora for model training and the improvement of modelingmethods. In principle, the speech corpora contain speech in individual languages by pairs of
one male and one female who are voice actors
or other occupational speakers. We adopt neural networks for the text-to-speech models andare conducting R&D while balancing soundquality and speed improvement. We focus on
using general-purpose CPUs to overcome theeconomic obstacle for commercial implementation which may arise if text-to-speech services require expensive GPGPUs.
One of the neural networks that we areworking on has the structure as shown in Figure
2. The computation speed is RTF = 0.4 (RealTime Factor; the ratio of the computation speed
to the length of speech time). The sound quality
is MOS = 4.3 (Mean Opinion Score; subjective
evaluation value on a scale of five), a value
close to around MOS = 4.5 of a human voice.
When applying text-to-speech as the output of automatic simultaneous interpretation

services, the synthesized speech is sometimes
adjusted to play faster than normal speed toshorten the delay between the indication of
the translated text and the completion of thesynthesized speech. Stretching or shortening
the whole speech depending on the elasticityof phonemes is a well-known method. When
we applied this to the aforementioned neuralnetwork, the results of subjective evaluation
showed that the sound quality does not deteriorate significantly within the stretching rate between 0.75-1.25 times the original speed.

■Speech Recognition and Text-toSpeech for Daily Conversations
As for the 15 primary languages stated in the
GCP 2025, we have mostly completed our missions for speech recognition and text-to-speech,
and have published the results via VoiceTra®,a speech translation app released as a field experiment, and have commenced the granting ofcommercial licenses.
Under the supplementary budget for
FY2021, it was decided to newly support Russian, Arabic, Hindi, Italian, and German. We
created speech corpora and developed models
for these languages in FY2022, and with a few
exceptions, we released them on VoiceTra® bythe end of the same fiscal year. Additionally,in the same year, there was an urgent need tosupport speech recognition and text-to-speechfor Ukrainian in consideration of the Ukrainian

Figure 2

 eural network that can synthesize high quality
N
speech at a high speed

refugees. We intensively put efforts into the
procurement of the speech corpus and model
development, and as a result, we were able tosupport Ukrainian on VoiceTra® by August2022. It turns out that models were not the only
ones that were trained--the fact that we wereable to respond to a new language and completesocial implementation in such short period oftime, is the outcome of our long-term efforts on
multilingualization.

■Future Prospects
The GCP 2025 aims to achieve social implementation of automatic simultaneous interpretation technology, taking the opportunity of the EXPO 2025 Osaka, Kansai, Japanas a showcase. The EXPO will be held from
April 13, 2025, and there is less than two
years remaining as of the time of authoring
this article. We will arduously make further
efforts to improve the accuracy and speed ofspeech recognition, as well as the quality andspeed of text-to-speech in this short stretchof time.

NICT NEWS 2023 No.4

5

Aiming for a World of Mutual Understanding with AI

Table 1

Automatic Simultaneous Interpretation Technology
Technology that automatically interprets spoken words without delayJapanese sentences divided into chunks and the results of theirautomatic translation into English

おはよう ござい ますGood morning.

Ｆ 誌 記者 の カーティスタッカーと 申し ますI'm Curtis Tucker, a reporter for F Magazine

プレゼン の 中 でIn the presentation.

この 製品 の 開発 に 七 年 かかっ
た こと に 言及 さ れ て い まし た

It was mentioned that it took 7 years
to develop this product.

特に ご苦労 なさっ た 点 は 何 だっ What do you think was the most difficult point?
た と 思わ れ ます かpeech translation has become quite
familiar thanks to the emergence ofvarious applications. However, at present,
sequential translation to translate speech
sequentially after a speaker finishes speaking is main stream. However, NICT has been
conducting R&D on simultaneous interpretation technology to have an application
simultaneously translate speech while aspeaker is speaking. This technology makes
it possible for a speaker to continue speaking without being aware of the intervention of an interpreter and also significantly
reduces the time lag in translation.

S

UTIYAMA Masao
Director of Advanced Translation
Technology Laboratory, Advanced
Speech Translation Research andDevelopment Promotion Center(ASTERC), Universal Communication
Research Institute
After completing graduate school, hejoined the Communications ResearchLaboratory (currently NICT) in 1999. He
has been engaging in research on natural language processing, especiallymachine translation. Ph.D. (Engineering)■ Background
Foreign visitors had decreased in number
due to COVID-19 but are increasing again.
Additionally, foreign residents are also increasing. Accordingly, it is expected that opportunities for communications with foreign
nationals will increase nationwide, including in local areas, and that people will facea language barrier in various scenes at work
and in daily living, such as in administrative
procedures or on occasions of receiving medical services, using transportation services,or enjoying sightseeing. Toward the World
Exposition (Osaka Kansai EXPO) scheduled
to be held in 2025, dramatic development of

automatic translation technology is expected
to realize the multilingual simultaneous interpretation system, which enables conversations with people around the world on areal-time basis. The technology is expected
to be utilized at pavilions and lecture venues.
Under such circumstances, NICT established
an R&D consortium together with other companies,*1 and has been conducting R&D on
automatic simultaneous interpretation technology under entrustment of the Ministryof Internal Affairs and Communications fora program, "Research and Development ofAdvanced Multilingual Translation Technology,"*2 in its project, "R&D Project for Information and Communications Technology(JPMI00316)."■ Outline of Automatic SimultaneousInterpretation Technology
Figure 1 shows the basic form of simultaneous interpretation by computer. [i] First,
a speech that was input is changed into text
using a speech recognition module and thetext is seamlessly input into the next module,
a divider. [ii] The text is divided by the divider module into chains of parts of words(chunks as translation units) that are suitedto translation as had been learned in advance.
[iii] Those chunks are converted into one ofmultiple languages by an automatic transla-

設置 手順 を 簡単 に する ため

In order to simplify the installation
procedure.

持ち運び できる 大き さ に する ことがYou should make it portable.

開発 当初 から 主 な 目標 の ひと
つ でし たIt has been one of the main goals
since the beginning of development.

tion module. [iv] They are output in sequence
as speech by a speech synthesis module.
Differences between simultaneous interpretation and sequential translation are as
follows.
• In the case of sequential translation, translation starts after a speaker finished speaking,while in the case of automatic simultaneousinterpretation, the system needs to decideon the timing of automatically starting interpretation.
• In the case of sequential translation, one
to several sentences are translated at once,
while in the case of automatic simultaneousinterpretation, it is necessary to do interpretation in a shorter unit (chunk) to reduce thetime lag.
• In the case of sequential translation, speech
as a whole must be translated without omission as it is, while in the case of automatic
simultaneous interpretation, it is sometimes
required to sort out the content of speechin order to promote listeners' understanding
and reduce the time lag.
Additionally, for both sequential translation and simultaneous interpretation: which
need to do translation or interpretation in
multiple languages, it is necessary to apply
technologies for dealing with those differences mentioned above for multiple languages.

■ Technology to Intermittently
Divide Chains of Input Words

Figure 1 O utline of automatic simultaneous interpretation technology (a significant difference from sequential
translation is the existence of a chunking module)

6

NICT NEWS 2023 No.4

In simultaneous interpretation, a speaker's speech is recognized in succession as achain of words, and dividing this chain ofwords into chunks is the first step. For thispurpose, we conducted research and developed a dividing module through machinelearning technology using a neural network.
In the machine learning, we (1) created training data: which designate where to divide an
input sentence, in multiple languages, and (2)created a dividing module that simulates theformer. Table 1 shows an example in which

73/8.9/*0:;</8*2=*>28?@*40*.*A8.0@B.A420*:04A
"$#)

")#(

")#)
!$#)

!(#(
!"#$

EC//D6*40*
F.C.0/@/G*
E628A/0/?*<H*.82:0?*&)I

!)#)
%$#)

%!#(*

%)#)

EC//D6*40*J09B4@6G*
E628A/0/?*<H*.82:0?*"&I%&#'

EC//D6*40*M640/@/G*
E628A/0/?*<H*.82:0?*$)I

%&#%*
,#%*

'#&*

+#"*

!&#'

%"#!*

%"#%*

%%#)*
+#(*

EC//D6*40*
L4/A0.;/@/G*
E628A/0/?*<H*.82:0?*"%I

EC//D6*40*K28/.0G*
E628A/0/?*<H*.82:0?*"(I

+#)*

$#)
)#)

-.

/0

C/8*@C//D6

Figure 212.:A2;.A4D*@/0A/0D/*@/9;.0A.A420

34

56.:A2;.A4D*D6:0D1409

Average number of words in a translation unit for multiple languages

a chain of words in Japanese is divided into
chunks and they are automatically translated
into English. Figure 2 shows the number ofwords per speech, the number of words byautomatic sentence segmentation, and thenumber of words by automatic chunking.
From this, it is found that the number ofwords in a translation unit decreases significantly only by dividing speech into sentences. For example, in the case of Japanese in
Figure 2, the number of words until a speakerfinishes speaking is around 23, but when dividing speech into sentences, translation canbe started earlier when the number of words
is around 13. Furthermore, when dividing
speech into chunks, the length of a translation unit becomes shorter than in the case ofdividing speech into sentences and the lengthis shortened by 30% to 50% depending on
languages. In this manner, by doing translation by units of chunks without delay, smoothcommunications can be promoted.

■S orting out of the Content of
SpeechIn the case of simultaneous interpretation,technology to shorten sentences in speech issometimes required. We also realized thistechnology through machine learning using aneural network. Through the use of the technology to shorten sentences, it is possible toplainly communicate the content of speech.
Furthermore, the results of speech recognition do not contain punctuation marks, andtherefore, we also conducted R&D on technology to insert punctuation marks. The
following examples are a sentence that is
a result of speech recognition, a sentence
wherein punctuation marks are automaticallyinserted, a shortened sentence, and a translated sentence.
〇 Example Japanese to English(Output of speech recognition) 先だって発信 し た 通り この 度 当社 製品 をより 広く 紹介 する ため 三か月 後 の展示 会 に 出展 する こと に なり ま
した(With punctuation marks inserted) 先だって発信した通り、この度当社製品をより広く紹介するため、三か月後の展示会に出展することになりました。
(Shortened sentence) 当社の製品をより広く紹介するために、3か月後に展示会に出展することになりました。
(Output of translation) In order to introduce our products to a wider audience, inthree months' time, we will be exhibitingat an exhibition.
〇 Example English to Japanese(Output of speech recognition) underwhat kind of criteria and when was theseismic capacity evaluation conducted in
kawanishi city
(With punctuation marks inserted) Under
what kind of criteria, and when was theseismic capacity evaluation conducted in
kawanishi city.
(Shortened sentence) What kind of criteria and when was the seismic assessmentin Kawanishi City?
(Output of translation) 川西市での耐震診断はどのような基準でいつでしたか?

■ Future Prospects
At present, automatic simultaneous interpretation technology is being developed
rapidly. Please expect that the multilingualinterpretation system that enables real-time
conversations with people all over the world
will have been made available by the time of
the 2025 Osaka Kansai EXPO.

＊1

h ttps://www.toppan.co.jp/news/2020/08/
newsrelease_200828_2.html (In Japanese)＊2  h t t p s : / / w w w . s o u m u . g o . j p / m a i n _content/000700925.pdf (In Japanese)

NICT NEWS 2023 No.4

7

Aiming for a World of Mutual Understanding with AI

Data-driven Intelligent Communication Technology

multimodal dialogue system for el-

A derly care (MICSUS) is one of the*1

OHTAKE Kiyonori
Director General, Data-driven
Intelligent System Research Center,
Universal Communication ResearchInstitute
After finishing a doctoral course in 2001,
he joined ATR spoken language communication laboratory. He joined NICT in
2006. He has been engaged in research
projects on natural language processing
and spoken language processing. Dr. ofEngineering

data-driven intelligent communication
technologies researched and developedby the NICT Universal Communication Research Institute (DIRECT). During FY2022,
large-scale MICSUS usability testing was
conducted across Japan from Hokkaido
in the north to Shikoku and Kyushu in thesouth. This article reports the final results ofthis testing and gives updates on our MICSUS R&D.

■Background
During NICT’s fourth medium-to-longterm plan implementation period, DIRECT
has researched and developed a technology
capable of performing in-depth semantic
analysis of text information available primarily on the internet (i.e., web-based information). We also worked to put this technology
into practical use. During the subsequent fifth
medium-to-long-term plan period which began in FY2021, we have continued to develop the web-based information analysis technology into a verbally interactive technology
able to engage in friendly, face-to-face conversation with its users for the purpose ofproviding them with helpful web-based in-

Figure 1

8

NICT NEWS 2023 No.4

formation and collecting specific information from them. We collectively call thesetechnologies “data-driven intelligent communication technologies.” MICSUS is a data-driven intelligent communication technology developed in collaboration with KDDI
Corporation, NEC Solution Innovators, Ltd.
and the Japan Research Institute, Ltd. This
project was funded by the Japanese Cabinet
Office’s Cross-ministerial Strategic Innovation Program (SIP) during the program’s second period (FY2018–FY2022).

■MICSUS Objectives
Japanese society has been rapidly aging
in recent years due to declining birth rates
and economic difficulties. This is causing aserious social issue: the lack of an adequate
workforce to provide nursing care to seniorcitizens. While demand for nursing care is
growing, the amount of financial support
and the number of laborers needed to sustain
services are falling short. Although it is estimated that 253,000 nursing care workers will
be needed by 2025, this demand is unlikely
to be met. Japan faces a shortage of 377,000
workers.*2 Traditionally, care managers—akey player in nursing care services—evaluate the health of elder care recipients throughin-person interviews. MICSUS is designed to

MICSUS usability testing in action in Hidaka, Kochi Prefecture

perform some of these care managers’ tasks,
thereby reducing their workload. In addition,
MICSUS is intended to offer elderly people
more frequent communication—a vital activity in maintaining health—through casualconversation with them using web-based information.
For details about the structural and technological aspects of MICSUS, please refer to
the NICT Journal article published in 2022
(Vol. 68, No. 2, pp. 141–149)*3. In this report, we mainly focus on the results of largescale MICSUS usability testing carried out in
FY2022 which are not included in the previous article.

■ Large-scale MICSUS Usability
Testing with Elderly People
Large-scale usability testing was conducted across Japan from June 2022 to January
2023. 179 elderly people participated. A total of 927 conversations consisting of 26,704
speech exchanges between MICSUS andelderly subjects (who were responsible for
12,885 of the remarks) were recorded during
the testing period. The cumulative durationof these conversations was approximately
95 hours and individual conversations lasted approximately six minutes on average. A
subject spoke approximately 14 times on average during a single conversation. These indicate that elderly MICSUS users conversed
for significantly longer periods of time thanelderly people who used other related technologies (i.e., smart speakers). An elderly
lady in Hidaka, Kochi Prefecture, one of
the usability testing participants, used MICSUS daily for 15 days. A video showing her
conversing with MICSUS can be viewed athttps://www.youtube.com/
watch?v=cuIHPYEqOKc&t=75s (in Japanese)(Figure 1). We evaluated the nearly 13,000
remarks made by elderly people recorded
during the nationwide experiments—one of
the most extensive sample sizes for this typeof research ever conducted in Japan. We
asked three third-party evaluators to examine

the legitimacy of each remark by reviewing
the entire conversation in which it appeared.
When their opinions were divided, they took
a vote. For example, they determined whether a subject responded with “yes,” “no” or “I
don’t know” in answering MICSUS’ question, “Are you eating three meals daily?”
When MICSUS’s semantic interpretation of
its users’ responses was compared with thatof the human evaluators, its interpretation
accuracy using its deep learning capability
was found to be 93.5%. In addition, 92.9%
of MICSUS’ casual conversation responses
to its elderly users and its communicationin relation to other services were judged tobe appropriate. During casual conversations,
91.8% of MICSUS’ 1,346 responses made using web-based information were found to beappropriate. Furthermore, 25% of MICSUS’
answers made elderly users smile and morethan half of elderly subjects found conversing
with MICSUS interesting. These results suggest MICSUS’ potential effectiveness in offering elderly people more frequent communication. We believe that these results would
be difficult even for humans to achieve andindicate MICSUS’ ability to hold high-quality conversations. We also compared the time
it takes for care managers to check the healthof elderly people with and without the assistance of MICSUS. We found that the use of
MICSUS can reduce the amount of time caremanagers spend on health checks by 69% or
more. In addition, we conducted a survey of120 elderly people who participated in the
MICSUS experiments. The average rating of
the overall MICSUS experience was 4.2 ona scale of 1 to 5, with 5 being the best experience. When we interviewed elderly users,
their comments included, “Conversation withMICSUS was very enjoyable and I was eventempted to joke around with it,” and “Talking
to MICSUS encouraged me to use my mind.”■ Future Prospects
We may consider integrating a largescale generative AI-based language model,such as OpenAI’s ChatGPT, into MICSUS’casual communication function in the future. However, generative AIs are known
to sometimes produce false responses whenanswering questions (i.e., they may lie). Our
assumption is that most MICSUS users are
elderly people with relatively low levels of
information literacy. To ensure the credibility of the information MICSUS provides, we
may program it to orally inform users of the
URLs of websites from which it derives information. We will also encourage MICSUS
to frequently warn its users to use web-based
information carefully, as the risk of obtaining
false information from the internet cannot
be completely eliminated. We are currently
against incorporating a large-scale generative AI-based language model into MICSUS
because this would make its use significantly
more costly due to the extensive resources
needed to run such models. During the usability testing, MICSUS held casual conversations using a medium-sized web-based
information database prepared in advance.
This scheme proved effective in enabling
MICSUS to hold proper conversations without performing intensive deep learning tasks.
Cost efficiency is an important consideration,
since elderly people will be MICSUS’ primary users. In addition to the MICSUS project,
DIRECT is researching and developing agenerative AI-based dialogue system capable of formulating hypotheses using the hugeamount of information available on the internet and providing them to its users alongwith the hypotheses’ factual bases. Our goal
is to make this system capable of facilitating
brainstorming by users to address challenging issues.
＊1  MICSUS: Multimodal Interactive Care SUpport
System＊2  M inistry of Health, Labor and Welfare, Press
Release, ”Demand/supply Estimation of elderly
care workforce”
https://www.mhlw.go.jp/stf/houdou/0000088998.
html (in Japanese)＊3   N I C T “ J o u r n a l o f N I C T ” . V o l . 6 8 , N o . 2
https://www.nict.go.jp/publication/
shuppan/kihou-journal/houkoku68-2_
HTML/2022U-03-05.pdf (in Japanese)

NICT NEWS 2023 No.4

9

Aiming for a World of Mutual Understanding with AI

Smart Data Analytics Technology
Predictive analysis and behavior support through sensing data collaboration

Figure 2
Figure 1

smarter, more sustainable society
could be developed through integrated use of various types of sensory data.
This approach to data utilization could
enhance comprehension of complex real-world events and allow optimal actions
to be taken in specific circumstances. We
have been developing a smart data analytics
technology capable of integrating different
types of sensory data and making it available
for prediction and analytical purposes. Our
specific R&D projects include: 1) the development of cross-data analysis techniques
which can be used to collect various types ofreal-world sensory data and identify, analyze
and estimate correlations between events
across different disciplines; 2) developmentof a decentralized, federated AI technology
for constructing prediction models designed
to help people comprehend and respond
appropriately to real-world events using
data obtained anonymously from different
individuals; and 3) construction of an xData(cross-data) platform into which the foregoing technologies will be incorporated. In
addition, we are working with our collaborators to tackle various issues using the xData
platform.

A

ZETTSU Koji
Director General, Big Data Integration
Research Center, Universal Communication Research Institute
After completing graduate school,
he joined NICT in 2005. He has been
engaging in research on data mining,machine learning, and databases.
Doctor of Informatics■Background
Global climate change and growing population disparities between urban and rural areas in recent years have prompted the adventof complex urban issues related to the environment, traffic, natural disasters and energy. To
cope with these issues, the Japanese government formulated a comprehensive data strategy with the goal of making society smarter andmore sustainable. This strategy states that thekey to creating new value is twofold: integrated use of a wide variety of data and construction of platforms to analyze integrated data
for the purpose of creating and offering helpful services. A cyber-physical system (CPS)
combines various types of data from physicalspace, analyzes it using AI techniques to com-

10

NICT NEWS 2023 No.4

prehend real events appropriately and suggests
proper actions people can take under specificcircumstances. These CPS functions can beused to meet comprehensive data strategy objectives. Our research has therefore focused on
developing a smart data analytics technology
as a CPS component.

■Cross-data Analysis Techniques
We have been researching and developing
a multimodal AI technology capable of performing cross-data analysis by identifying andestimating difficult-to-detect correlations between different types of data. This technology
is able to discover temporal and spatial associative patterns which may exist in data collected from various physical space sources using a convolutional recurrent neural network(CRNN), a graph neural network, etc. Discovered associative patterns have been used to
predict short-term changes in environmental
quality (e.g., using an air quality index (AQI))
in localized areas. We have also developed an
MM sensing technology capable of predicting
traffic hazards (e.g., traffic-related air pollution
and dangerous driving conditions) and AQI in
surrounding areas for drivers using image log
data captured by portable and fixed cameras.
It supports drivers by predicting traffic pollution and driving risks (near-misses, etc.) using
dashboard cameras, etc. (Figure 1). To improve
the prediction performance, we have developed a cross-modal attention model that canaccurately predict correlations at the event-relationship level as well as at the conventionalfeature space level by employing multiplex
attention (a machine learning mechanism thatdynamically identifies key data) to object elements extracted from different types of data.
The model received a high score in international benchmarking*. Furthermore, we havebeen researching zero-shot/few-shot learning
methods which can be used to make prediction models adapt to new situations even when
only very small amounts of relevant training
data are available.

Support for traffic pollution control in an ASEAN regionE arly warning of photochemical-oxidant pollution based on
short-term environmental quality prediction (collaboration
with environmental monitoring service providers)■Distributed Federated AI
Technology
Centralized AIs—which use cloud platforms to perform all necessary tasks fromdata collection to machine learning—have
traditionally been the most widely used AI
resources. However, this form of AI is becoming increasingly problematic because of
growing global concern about privacy anddomestic data protection. To address this issue, we have been researching and developinga federated learning technology that enables
distributed learning by multimodal AI models
while maintaining data in individual edge environments.
Since data collected on events at the edge
will vary in terms of area, duration, frequency
and density, we are developing methods to aggregate models to account for these variations
and improve the performance of federated
learning. Moreover, since available computing resources change dynamically in the edge
environment, we are developing a distributed
learning method that can flexibly respond tothese changes.

■ xData Platform
We are constructing an xData platform
which will allow us to develop smart applications using cross-data analysis technology.
This platform stores hundreds of terabytes
of event data extracted from sensory data invarious fields, such as environmental, transportation and health data. The platform isequipped with a number of application programming interfaces (APIs) to facilitate various tasks—data collection and integration,correlation analysis and prediction using data
mining, multimodal AI, etc., generating maps
to graphically represent analytical results, issuing alerts and searching for optimal routes.
We are also developing xData Edge, a tool

Figure 3

 rediction and analysis of driving risk events using MM sensing (collaboration with transportP
companies)that will provide users with more secure analysis and prediction based on their data usingfederated learning and other techniques. We
will also equip the xData Platform with information assets that package data models,prediction models, processing programs, sample data, development manuals, etc., for eachfield in which cross-data analysis is applied.
The functions and information assets of thexData platform are also deployed in the Data
Centric Cloud Service (DCCS) of the NICT
Testbed to promote application development
and verification experiments as a platform fordata and service collaboration and application
development.

■Practical Application Examples
NICT’s short-term environmental quality
prediction information asset is able to analyze
a combination of environmental and meteorological data collected from various sources andperform short-term, fine-grained environmental quality prediction. Using the information
asset, an environmental monitoring company
is developing an application to provide early
warnings of high levels of photochemical oxidants (Figure 2)—the only air pollutant in Japan whose concentration continues to exceed
environmental quality standards. The company is using DCCS to customize the application
to meet the specific needs of different municipalities by collecting additional data, adjusting
training data, optimizing prediction processes
and adding post-processing functions in accor-

dance with municipalities’ workflows.
NICT’s MM sensing information asset
detects the trajectory and speed of surrounding objects from dashboard camera data, etc.
and estimates safe areas where collisions can
be avoided while extracting potential events
leading to driving risks and classifying eventsin detail according to the type of driving risk,
occurrence status, etc. A transport company
has developed an operation management application using the MM sensing information
asset (Figure 3). The application has been used
to analyze risk trends for individual drivers
and analyze risk factors in combination withenvironmental data and driver biometric datacollected from wearable devices, etc.

■Future Prospects
Based on the xData platform, we designed
cyberspace architecture for data distribution,
storage, analysis and prediction in Beyond
5G/6G and published it in NICT’s Beyond
5G/6G white papers, etc. In addition, we willcontribute to the realization of CPS in the beyond-5G/6G era through research and development on the connection between cyberspaceand physical space, edge AI for behavioral
support and digital twin collaboration using an
orchestrator.

＊

A C M M u l t i m e d i a G r a n d C h a l l e n g e o n
Detecting Cheapfakes

NICT NEWS 2023 No.4

11

File 25
History of Changes up to NICT NEWS 500th Issue
Press Office, Public Relations Department

his issue of NICT NEWS is the 500th issue counting fromthe first issue of its predecessor, Radio Research Laboratory (CRL) News. Here, we would like to look back on thehistory over nearly half a century.

T

The first issue of Radio Research Laboratory News, a PR magazine of the Radio Research Laboratory under the jurisdiction of the
Ministry of Posts and Telecommunications of the time, was published in April 1976 (Photo 1). 1976 was a year of political turbulence during which former Prime Minister TANAKA Kakuei wasarrested due to the Lockheed scandal. The first issue contained anarticle concerning acoustic influences of lightning discharge and introduced an experiment of underwater laserscope.
It was July 1984 that the 100th issue of the Radio Research Laboratory News was published (Photo 2). During this year, satellite
broadcasting was commenced in Japan. In the Olympic Games heldin Los Angels in July, YAMASHITA Yasuhiro, who is currently the
JOC president, won the gold medal in Judo (open-weight category),
and Carl Lewis of the United States won the gold medals in the
men's 100 meters and in the long jump.
The 200th issue was published as CRL News of the Communications Research Laboratory, the Ministry of Posts and Telecommunications, in November 1992 (Photo 3). This issue published immediately after the bursting of the bubble economy already warned about
young people's lack of interest in studying science and engineering
as a serious problem.
The 300th issue was published in March 2001 in color mode,
while changing size from the conventional B5 size to A4 size (Photo 4). It introduced demonstration experiments in which multiple
research institutes connected with a network observed the global
environment as virtual laboratories. During this period of time, theMinistry of Posts and Telecommunications became the Ministry ofInternal Affairs and Communications through the central government reform, and the 300th issue stated that CRL would start afreshas an incorporated administrative agency from the new fiscal year
starting from coming April.
The 400th issue was the New Year's edition for 2011 of NICT
NEWS (Photo 5). This issue featured a space weather forecast,which continues to be one of the important missions of NICT. On
March 11 of this year, the Great East Japan Earthquake occurredand caused tremendous human and economic damage, which is still
fresh in our minds.
During this half century, social circumstances have changed
and the forms and names of the organization have also changed.
However, the mission of NICT NEWS to communicate the latest
research outcomes in an easy-to-understand manner will never change. We hope that you continue to enjoy reading NICT
NEWS.

12

NICT NEWS 2023 No.4

●Photo 1
First issue of Radio ResearchLaboratory News
Upon launching Radio Research Laboratory News

Multiple Sound Spot Synthesis System: a Next-generation Technology
Capable of Simultaneously Synthesizing Different Sounds in Different Zones

History of
NICT NEWS
▼1976

Research projects in FY1976
Mr. Thunder's drums and deuteron whistler, and other topics

1984●Photo 2
100th issue of Radio ResearchLaboratory NewsIn commemoration of the100th issue
Various stories at the time oflaunch and other topics●Photo 3
CRL News (No. 200)
Upon publishing CRL NewsNo. 200

1992

First job was the editing ofnews and other topics

2001●Photo 4
CRL News (No. 300)
New year's remarks 2011
Special feature: Space weather
forecast
Topics: NICT selected as
ICSU-WDS International Program Office, and others

Multimedia virtual laboratory
demonstration experimentsin the geoenvironmental
science fieldA flying workstation andother topics

Senior Researcher, Advanced SpeechTechnology Laboratory, Advanced
Speech Translation Research andDevelopment Promotion Center(ASTERC), Universal Communication
Research Institute

W

Changes up to CRL News No.
200●Photo 5
NICT NEWS (No. 400)

OKAMOTO
Takuma2011● Biography
1981 Born in Numazu City, Shizuoka
Prefecture
2004 Earned a bachelor's degree
in Electronics and MechanicalEngineering from Tohoku
University
2006 Completed a master’s programin System Information Sciencesat Tohoku University’s Graduate
School of Information Sciences
2009 Earned a Ph.D. in System
Information Sciences fromTohoku University’s Graduate
School of Information Sciences
2009–2012
Worked as a postdoctoral
researcher at Tohoku University
2012 Joined NICT
2020 Current Position
2003 Appointed as a Visiting
Associate Professor, Tohoku
University● Awards
2012	32nd Awaya Prize Young
Researcher Award
2018 57th Sato Prize Paper Award
2022 9th Society Activity ContributionAwardhen voices or audio recordings areplayed through loudspeakers, they
are propagated in all directions. This causesinterference when multiple voices and/or audio recordings are produced simultaneously,
making it difficult for listeners to identify specific sources. To address this issue, we havedeveloped a multiple sound spot synthesis
system, consists of a circular array of 16 loudspeakers able to simultaneously synthesize
different sounds in different, targeted areas.
This allows sound transmitted to non-target
areas to be cancelled out using antiphasesound signals. In addition, this system is expected to be able to simultaneously deliver
different audio recordings in different directions without interference.
I previously carried out two projects—one on
multilingual speech synthesis and anotheron sound field control—funded by the JSPS
KAKENHI Grants-in-Aid. I am now integrating
the results of these two projects with the goalof putting the multiple sound spot synthesis
system into practical use. This project has
been conducted in conjunction with PoCC
launched as part of NICT's IDI. HIKITA Keita(NICT Europe Center), SUDO Miyuki (NICT
General Affairs Department) and KUWAHARA
Manae (currently working at an NTT facility in

Q&As

Q What is good things to be a researcher?
A As a researcher, I’m thrilled when my research proposals/manuscripts are accepted. I would be very honored if myresearch publications could impact the research community
around the world and if my research achievements could
benefit the public. Aside from work, I enjoy having drinks
with other researchers and collaborators.

Q What are you interested in other than research?

A Jogging 10 km is my daily routine. Research ideas often

come to me while running. I’m very happy that I can finallygo out for drinks with my colleagues and friends again after
a three-year hiatus due to the COVID-19 pandemic.

Q How do you spend your time on holidays?
A I jog and take my child to soccer practice in the morning.
My family and I then go shopping in the afternoon. I often
join my neighbors as they play with their children in a
nearby park. I also enjoy eating out and going on overnight
trips with my family.

USA)—young NICT general staff members inmanent exhibition at NICT headquarters
volved in the PoCC project—have also been
(exhibitor: YASUI Yuri, NICT Public Relations
giving me a help with this technology inteDepartment) and invited talks. I’m pleased
gration effort.
to have received many positive comments
I’ve made progress achieving practical use
from the general public at these events. I’mof the multiple sound spot synthesis system
now working to integrate the real-time multhrough many events. These include ontilingual speech translation system develsite demonstration of the portable multiple
oped by ASTREC and the multiple sound spotsound spot synthesis system, acquisition ofsynthesis system. This integrated system wasexternal funding, construction of a website
demonstrated at a NICT Open House event indesigned to publicize the system, usability
June 2023.
testing at the National Museum of EmergI will continue my efforts to put the system
ing Science and Innovation, exhibition andinto practical use in collaboration with otherdemonstration of the system during the genresearchers and cooperators.
eral conference of the Institute of Electronics, Information andCommunication Engineers(exhibitor:
ISHII Kensuke, NICTASTREC), exhibitionand demonstrationof the system during
the G7 meeting in
Gunma Prefecture,(exhibitor:
SORA
Wakana and TANIOKA Daisuke, NICT
PublicRelationsFigure Multiple Sound Spot Synthesis System
Department)
perNICT NEWS 2023 No.4

13

NICT researcher
receives 2023
Japan Prize●Area of Award: Electronics, Information and Communication●Awarded for

Significant contributions to the development of long-distance, large-capacity optical fiber networks,
mainly through the development of a laser diode pumped optical amplifier

The Maejima Hisoka Award was established in memory of Mr. Maejima, one of the founders of Japan’s telecommunications industry, and to pass on
and promote his spirit. The award is presented to those who have made an outstanding contribution to the advancement of the information and communications industry (including postal services) or the broadcasting industry. One group of NICT researchers received this award in FY2023.

68th Maejima Award

Senior Expert, Advanced ICT Research Institute/
Director of Nano-scale Functional Assembly ICT Laboratory, Kobe Frontier ResearchCenter, Advanced ICT Research Institute

YAMADA Toshiki

Senior Researcher, Nano-scale Functional Assembly ICT Laboratory, Kobe Frontier
Research Center, Advanced ICT Research Institute

KAJI Takahiro

Senior Researcher, Nano-scale Functional Assembly ICT Laboratory, Kobe Frontier
Research Center, Advanced ICT Research Institute● Awarded for: Research and development of organic electro-optic polymers and ultrahigh-speed light control technology● Date: April 6, 2023
Receivers’ Comment: We are very honored
to receive the Maejima Hisoka Award. We appreciate the recognition it brings to our longefforts, including the development of organic
electro-optic polymer materials and practicalultrahigh-speed light control devices. Our heartfelt thanks go out to the many people who havehelped and supported our research activities.
A series of fundamental technologies we havebeen developing for many years has finally comeFrom left, KAJI Takahiro, YAMADA Toshiki, and OTOMO Akira

together to create products with potential practical applications. This has led us to launch joint
research projects with many private companies.
We will work with these companies to put products into practical use.

ISSN 2187-4050 (Online)

OTOMO Akira

URL: https://www.nict.go.jp/en/

Tsushinbunka Association

4-2-1 Nukui-Kitamachi, Koganei, Tokyo
184-8795, Japan
TEL: +81-42-327-5392    FAX: +81-42-327-7587

Receiver’s comment
Optical fiber communications technology is basically composed of optical fibers, laser diodes, photodetectors and opticalamplifiers. I was involved in the development of an erbium-doped fiber amplifier (EDFA), a very compact optical amplifier designed to achieve broadband transmission, noise reduction and high signal output without oscillation. As transmission signals enter an EDFA, the input optical signal (1.55 μm
wavelength range) is coupled with excitation light (1.48 μm or 0.98 μm in wavelength) before passing through erbium-doped fibers. An EDFA is a very
simple optical circuit equipped with an optical multiplexer/demultiplexer and optical isolators. Its greatest advantage is its excellent connectivity andcompatibility with optical fibers through which optical signals are transmitted.
In 1988, ITU adopted the synchronous digital hierarchy (SDH) as a global standard technology for time division multiplexed transport networks. Consequently, an optical transmission rate of 2.5 Gbit/s was employed across the globe, including Japan. The intensity modulation/direct detection (IM/DD) format has since been used in the modulation and demodulation of optical signals within tiny devices, including laser diodes and avalanche photodiodes (APDs). At university, my research aimed to achieve use of the optical wavelength range in communications in addition to other wavelengths.
At the time, I thought it would be impossible for laser diodes to generate 200 THz optical waves, which have a spectral purity as high as microwaves
generated by microwave oscillators. I believed that the newly adopted IM/DD format could be used to reduce sensitivity to phase fluctuations, make theparticle properties of light very effective and easily improve the performance of laser diodes. Based on previous cost and effect studies, it was decided
that the transmission capacity of next-generation terrestrial communications should quadruple. However, achieving this target was expected to be very
challenging because of the requirement that existing communications relay infrastructure—including telecommunications stations and optical fiber
devices—had to be used without updating it. Quadrupling transmission capacity using the existing facilities required improving the fundamental S/Nratio to the minimum of 6 dB. Some researchers started presenting their results at scientific meetings to convince others that optical coherent communications systems could be a viable alternative to the existing system. I had been unproductively struggling to reduce thermal noise at the receiver sideusing a laser-diode amplifier. This motivated me to seek collaboration and outside expertise regarding the erbium-doped fibers. I began experiments incoordination with Professor Nakazawa’s research team. During these efforts, modifications I had previously made to improve a laser diode amplifier led
to a major breakthrough. In the subsequent optical transmission experiment—the world’s first of its kind—I set a new coherent transmission record (theFebruary 1989 Optical Fiber Communication Conference). A frequency division multiplexed EDFA with a frequency range approaching 10 THz divided
into 80 channels at 100 GHz intervals has since been put into practical use.
This EDFA with digital signal processing capability is able to correct not only
History of R&D of ultra large-capacity optical communication systems
phase fluctuations induced by a laser diode but even transmission distortion.
40 Tbit/s signals can be relayed using only a single EDFA unit in combination
with a multilevel transmission scheme. The use of EDFAs has enabled energy-efficient long-distance transmission (10,000 km across the seafloor andmulti mode fiber hollow core fiber
1,000 km or more on the ground) with only minimal delay (figure). This is atremendously beneficial achievement both technologically and economically.
I spent a majority of my time researching peripheral technologies that could
bring out the high-performance potential of erbium and take advantage of its
outstanding physical properties. This effort was made in collaboration withsingle mode fiber
many other Japanese and overseas scientists and the Japanese government.
When I completed the development of EDFAs in 1995, achieving an opticaltransmission rate of 10 Gbit/s, many people in Japan and elsewhere doubted
coaxial cable
the usefulness of such large transmission capacities. I am now glad to have
Fortune: Almost three generation (30 years) of history, folks has chattered "You're still on it". However I was supported by real demands.
created this technology.

NICT NEWS 2023 No.4 Vol.500

Information and Communication）

Published by Public Relations Department, National Institute of Information and Communications Technology
Issue date: Jul. 2023 (bimonthly)●Receiver: HAGIMOTO Kazuo  Principal Researcher, NICT（Program Director of Advanced Research in

Photo provided by The Japan Prize Foundation

NICT Principal Researcher HAGIMOTO Kazuo and his research collaborator,
NAKAZAWA Masataka (Professor Emeritus, Tohoku University), received the2023 Japan Prize. The prize ceremony was held at the Imperial Hotel Tokyo on
April 13, 2023 and was attended by the Emperor and Empress of Japan.