4‑3

4 Innovation Projects Innovation Initiatives

Innovation Projects Utilizing 5G Technology
OHTO Yutaka

KINOSHITA Shingo

FURUNO Noriyuki

We planned and implemented a project utilizing “5G” (5th generation mobile communication system) one of the mostpromising technologies of the future “5G” has the features of “high speed large capacity” and “low latency” compared toconventional systems and we utilized these features to develop three measures to realize “a new style of spectating”
Unfortunately there were no spectators at the venues but the experience was well‑received especially by those involvedin the Games This new style of spectating including its application to other competitions is expected to take off in the
futureKeywords：Innovation 5G High‑speed Large capacity Low latency Highly realistic New style of spectating

1．Introduction
The Tokyo Organising Committee of the Olympic andParalympic Games has made several efforts to make the

new changes in industrial structures and its introduction is progressing worldwide The introduction of thesetechnologies is progressing around the world In Japan
commercial services were launched in 2020

Olympic and Paralympic Games Tokyo 2020 (herein-

The measures utilizing “5G” the focus of worldwide

after referred to as “Tokyo 2020 Games”) “the mostattention were made possible by the generous support

innovative Games in history” To provide an “innovativeand cooperation of associated partners Although the

new spectator experience” we have been working toprojectʼs goal was to “provide a new spectator experi-develop and realize measures that utilize one of the latestence” and unfortunately there were no spectators ; thetechnologies “5G”

project was able to appeal to Games stakeholders and the“5G” is an abbreviation for the 5th generation mobile

media

communication system a next‑generation communica-

This article describes three measures planned andtion standard used for cell phones and other communica-

implemented by the “TOKYO 2020 5G PROJECT”

tions Compared to “4G” which is currently in wide-(Figure 1)

spread use it has features such as “high speed and large

spectator experience utilizing “5G”

a project that provides a new sports

capacity” and “low latency” For example the maximum

Although there were many challenges in realizing thetransmission speed is expected to be10 Gbit/s with aservice each challenge was cleared through coordina-delay of about 1 ms (1) The combination of “5G” withtion with the Games stakeholders and the measuresother technologies such as AI (Artificial Intelligence)

were implemented mainly at three venues Among theand IoT (Internet of Things) is expected to bring about

features of “5G” the “high speed and large capacity”
measures for “Sailing” and “Golf” and “low latency”

OHTO Yutaka
Logistics Planning Department DOCOMO CS IncKINOSHITA Shingo
NTT Human Informatics Laboratories NIPPON TELEGRAPH AND TELEPHONE CORPORATIONFURUNO Noriyuki
Tokyo 2020 Business Office NTT DOCOMO Inc

The Journal of The Institute of Electronics Information and Communication Engineers Vol 105 No 8 Supplement pp 211‑219 August 2022
© 2022 The Institute of Electronics Information and Communication Engineers

measures for “Swimming competitions” were designed
to make the most of these features The measures for“Sailing” and “Golf” were designed to take maximum
advantage of “high speed and large capacity” and “lowlatency” for “Swimming competitions” The project was
implemented under the following structure

Special Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 2020

211

Figure 1

Figure 2

Example of Live Video Transmission Via Offshore
Wide‑vision Display (©2021‑International Olympic
Committee‑All Rights Reserved)

Figure 3

Example of Video Transmission to Main Press Centre(©2021‑International Olympic Committee‑All Rights
Reserved)“Tokyo 2020 5G PROJECT” Logo (©2021‑International
Olympic Committee‑All Rights Reserved)

Organizer : Tokyo Organising Committee of the Olympic
and Paralympic Games
Cooperation : Japan Broadcasting Corporation (NHK)The Japan Commercial Broadcasters Association (JBA)
Technical cooperation : Intel Corporation Nippon Telegraph and Telephone Corporation NTT DOCOMO
INC

2．Description of the Measures
21

5G×Sailing×Ultra‑Realistic

Communication

Technology Kirari!
211

Overview of our Initiatives at the Sailing
Regatta

In sailing races

the event)
spectators usually have to use

binoculars to watch the races (from breakwaters etc )The ultra‑wide video images were also transmitted

because the races are held quite far from land At thelive to Tokyo Big Sight where the Main Press Centre

sailing regatta we took up the challenge of solving this(MPC) was located to convey the realism of the event

problem by using communication technology to create

to media personnel whose movements were restricted

an innovative style of spectating that goes beyond thewithin Japan because of coronavirus disease 2019

real world ; namely the spectators feel as if they were

(COVID‑19) pandemic (Figure 3)

watching the races from a special seat on a cruise ship

Although the sailing regatta was held without

Ultra‑wide video images of the races (with a horizontal

spectators we implemented a measure called a “virtual

resolution of 12K) were transmitted live to a 55‑m‑widestand” for sending support to the athletes from their far‑“offshore wide‑vision” display near the spectatorsʼ seatsaway families and friends (Figure 4)(Figure 2) by using the 5G and our ultra‑realistic
communication technology Kirari!

212

Initiatives at the sailing regatta for the TOKYO 2020
5G PROJECT are summarized below :Configuration of the Video Transmission
SystemThe configuration of the video transmission system forthe sailing regatta is shown in Figure 5 First the sailing･

Period : July 25 2021 to August 4 2021

action is captured from boats or a drone equipped with･

Venue : Enoshima Yacht Harbour

multiple 4K cameras The captured video images are

Kanagawa

Prefecture･

Total number of participants : about 4 000 (onlyathletes and related people were allowed to watch

212

synthesized as ultra‑wide video images with 12K
resolution and transmitted in real time by using Kirari!
and 5G

which enables highspeed

large‑capacity

THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERSmatrix is calculated from feature points extracted from areference still image

Next

during the synthesis

process video frames were combined using a projection‑transformation matrix calculated for each video
frame to correct disparities and find appropriate seamlines in the video images (i ea process called “seamsearch”)②

Ultra‑realistic Media‑synchronization Technology (Advanced MMT) (3)The video images generated using the ultra‑wide
Figure 4

Remote Cheering from the Virtual Stands (©2021‑
International Olympic Committee‑All Rights Reserved)

video‑synthesis technology are divided into multiple 4K
images for transmission so that they can be handled bygeneral display devices encoders and decoders In thiscase the image on the display was 12K wide and 1K

transmission The transmitted images were displayed inhigh so it was divided into three 4K images

real time on a 55‑m‑wide offshore wide‑vision display
floating in front of the viewing area at the regatta venue()

Wireless Communication Technology

The races of the sailing regatta at the Olympic and213

Technology(

Ultra‑realistic)

Kirari!

Paralympic Games Tokyo 2020 were held on six race
Communication

Technology(2)

courses (Figure 6)

Ultra‑wide video images were

captured from three camera boats for the Enoshima

In this project we implemented two technologicalcourse and ultra‑wide images were captured using aelements of Kirari! video transmission using ultra‑wide

drone camera in addition to three camera boats for thevideo‑synthesis technology and ultra‑realistic media‑

Zushi course

synchronization technology
The configuration of the network used in this project is①

Ultra‑wide Video Synthesis Technology

shown in Figure 7 For uplinking video from the boats toFirst the optical axes of each of four cameras are

shore and downlinking from the video‑editing base toaligned as much as possible and the cameras are set up

the barge‑mounted offshore wide‑vision LED display

so that their shooting ranges overlap adjacent captured

NTT DOCOMOʼs 5G line was used as the core inimages by 20‑40% Before the actual synthesis process

conjunction with multiple wireless communications withas a pre‑processing step to correct the disparity

different characteristics such as bandwidth distance

between each cameraand directionality NTT DOCOMOʼs 4G line was used fora projection‑transformation

Figure 5

Overview of the Video Transmission System (©2021‑All rights reserved NTT)

Special Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 2020

213

Figure 6

Map of the Race Courses for the Sailing Regatta (©2021‑All rights
reserved NTT)

Figure 7

Network‑configuration Diagram (©2021‑All rights reserved NTT)

flight control of the drone backup video and control ofthe server located on the camera boats

214

Results

Although the regatta was not attended by spectators
hundreds of people mainly athletes and related support

For the wireless equipment it is necessary to keep the

people watched the video transmission every day of theantenna surface facing the drone all the time To ensure

regatta We received comments from many people ofmore‑stable wireless communication we developed athose that expressed their surprise excitement andtechnology that automatically controls the direction ofhopes for the future of sailing Some comments were “I

the antenna by using real time kinematic (RTK) GPS

was surprised by the high sense of realism that I have

(Global Positioning System) information so that thenever experienced before” “It made it easy to under-

antenna always points in the direction of the drone

stand the race situation” and “If this style of spectating
becomes common

214

the number of sailing fans will

THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS

increase rapidly” For our aim of creating “a sense ofrealism as if the race was being held in front of you” wewere able to create the experience of “warping into therace space itself” by fusing the powerful actual‑size race
images displayed on the offshore wide‑vision display
floating on the sea with the actual surface of the sea sky
and other real spaces
215

Conclusion Remarks

Using commercial 5G and our in‑house ultra‑realistic
communication technology Kirari!

we successfully

Figure 8

Image of a Spectator Wearing an AR Device at a
Swimming Competition (©2021‑All rights reserved
NTT DOCOMO)

Figure 9

An example of the Image that Can Be Viewed with an
AR Device (©2021‑International Olympic Committee‑
All Rights Reserved)

transmitted live ultra‑wide video images (with 12K
horizontal resolution) via a 55‑m‑wide offshore wide‑
vision display installed at the Enoshima Yacht Harboursite and a similar but smaller display at the MPC at
Tokyo Big Sight in Tokyo At both sites the live video
transmissions were highly evaluated by athletes officials

and international media

The communication

technology used in this project will enable live viewing
not only within a competition venue but also to remotelocations including overseas ones Even during the
COVID‑19 pandemic this project can be ranked as ashowcase for achieving a remote world that NTT isaiming for From now onwards we will promote further
research and development on communication technologies mainly in the fields of sports and entertainment
using 5G and the All‑Photonics Network of IOWN
(Innovative Optical and Wireless Network) as an
infrastructure
･
22
221

5G×Swimming Competitions

Games)

August 27 to 29

2021 (Tokyo 2020

Paralympic Games)

Project Overview
the means to obtain･“competition information” such as athlete information･

In swimming competitions

Period : July 25 to 27 2021 (Tokyo 2020 Olympic

Venue : Tokyo Aquatics Centre
Total number of participants : Tokyo 2020 Olym-“times and standings” for spectators watching on‑site is

pic Games : 129 people

limited to electronic bulletin boards

Games : 242 people (spectators were athletes andmaking it achallenge to provide appropriate information To solve

Tokyo 2020 Paralympic

stakeholders only)

this problem we have realized a measure to transmit
athlete information standings times and other informa-

222

System Configuration

tion in real‑time to Augmented Reality (AR) devices

The following is an overview of this project First

worn by spectators via low‑latency communication

stats data such as the times of each athlete and athleteusing 5G so that they can obtain competition information

information are obtained processed by the AR system

without changing their line of sight and feel a sense of

prepared by the project processed into UI content andpresence and unity at the competition venue Figure 8transmitted to the network The contents are transmit-

shows an image of a spectator watching a competition

ted in real‑time via 5G to spectators wearing AR

and Figure 9 shows an image of the user interface (UI)

glasses Up to now when such content was transmitted

that can be viewed on the AR device

over a telecommunications network delays inevitably
occurred for example when information to display the

An overview of this project is given below

world record line is transmitted it is delivered to the AR

Special Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 2020

215

Figure 10

System Configuration (©2021‑All rights reserved NTT DOCOMO)

glasses later than the actual time making it extremely

past rather than the world record line on the AR

difficult to match the world record line with the athlete

glasses)seen through the AR glasses It was challenging tomatch the world record line with the athlete seen

224

through the AR glasses The use of “5G” in this project

The measures were implemented without any signifi-

reduced the delay to nearly “0” so the time gap between

cant problems during the Games period Although thethe actual race and the displayed content was virtuallyGames were held without spectators and for a shorteliminated allowing spectators to enjoy watching theduration a total of 371 people mainly parties related torace without any discrepancies in content such as thethe Tokyo 2020 Olympic Games and the Tokyo 2020

finish time

Results

Paralympic Games participated in the project Many
people commented “It will be a next‑generation way to

223

Technology

watch the games” “It was an excellent idea” “It was a

A commercial 5G base station was installed at thevery good idea to be able to see the world record line”

Tokyo Aquatics Centre where the swimming competi-and “The fireworks display when a world record wastions of the Tokyo 2020 Games were held and content

achieved was the best” etc We received many positive

was transmitted to the AR devices via the base stationcomments

The system configuration used in this project is shownand some people who experienced thesystem once said they would recommend it to others On
the other hand we also received comments such as “Iin Figure 10
Since 5G alone is insufficient to process stats datadonʼt want to do this because it would ruin my hair” “Myathlete information etc and ultimately deliver them toeyes get tired” and “I would like to see more images

the AR glasses the data was obtained directly at thefrom underwater”

venue rather than via the Internet to get as close toimproved in the future Regarding our goal “obtain thereal‑time as possible

necessary information as if you were watching TV atwhich indicate the issues to be

Future challenges include handling the data provided

home while feeling the presence at the venue” we(e g when the number of items increases) developing

believe that we were able to provide a new spectator

algorithms to select the required items from a large

experience based on the praise we received from theamount of data and process the contents without delay

many people who experienced the systemand developing more attractive contents in combination

Although we had some technical issues such as somewith other technologies (e g displaying e reproduction

missing stats data by creating the necessary informa-of the actual image of swimming competitions in thetion based on other information we produced results

216

THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS

that satisfied the spectators who experienced theAn image of the screen is shown in Figure 11

project Although there are still issues to be addressed in

An overview of the project is given below

the future we were able to achieve excellent results in･an environment that demands a very high level of
performance such as the Tokyo 2020 Games
225

Conclusion

Period : July 31 to August 1 2021 (men) August
6 to 7 2021 (women)･

Venue : Kasumigaseki Country Club･

Number of Users : Family lounge users : 90 (Actually the number was higher because some people

We provided a new spectator experience using AR
glasses by utilizing commercial 5G A total of 371 people

voluntarily tried the tablets (spectators wereGames stakeholders only)

experienced the system which was highly praised The
communication technology used in this project can be

232

System Configuration

applied within the same competition venue and in otherAmong the 18 holes we targeted videos from six holes

competitions In the future we expect further develop-

from 13 to 18 which were expected to be very exciting

ment of services by combining 5G with other technolo-

We obtained multiple videos (2K high‑definition im-

gies while taking advantage of its real‑time featuresages) for TV broadcasting installed at the venue andselected the footage that everyone most wanted to see

23
231

from among the multiple camera videos at each hole

5G×Golf

such as the athletes in action and transmitted them

Project Overview

Golf competitions are held on 18 holes so following

together over the network

This large volume ofoneʼs favorite player requires to walk a considerable

information was transmitted to tablets set up in the

distance and it isnʼ t easy to see what is going on at otherlounge using the high‑speed large‑capacity features ofholes The difficulty increases during competitions in the

5G Users could simultaneously view six videos fromsummer months such as July and August when theholes 13 to 18 and tap one of them to view the video ofTokyo 2020 Olympic Games were held Based in above

their favorite hole on a larger screen In addition

we took advantage of the large‑capacity transmission

information on the players such as their rankings andfeature of 5G to implement a measure that allows users

scores of player at the time

to watch high‑quality videos of competitions acrosssimultaneously for the enjoyment of the audience

multiple holes in a comfortable environment such as a

Furthermore live information such as “▲▲ player has aloungechance of birdie” was also created by project members

By operating a tablet installed in the lounge specta-

is also transmitted

and transmitted for the enjoyment of the audience

tors can view videos from multiple holes simultaneously

Figure 11

Image of the Screen (©2021‑All rights reserved NTT DOCOMO)

Special Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 2020

217

Figure 12

233

Network Configuration (©2021‑All rights reserved NTT DOCOMO)

Technology

A commercial 5G base station was installed at the

problems during the Games period

Although theimplementation period was short since the Games were

Kasumigaseki Country Club where the golf competition

held without spectators

for the Tokyo 2020 Games was held and users can enjoy

mainly to stakeholders over four days at the total duringservices in the lounge while receiving the signal

the Olympic Games including the final rounds of thethe service was provided

The configuration of the network used in this project ismenʼs and womenʼs competitions Many people com-shown in Figure 12 where camera images from the OBS

mented favorably on the service saying “Itʼs good to be(Olympic Broadcast Service) were obtained directly

able to choose which hole to watch” “Itʼs easy tofrom the venue and the video to be played from each

operate” “The videos are smooth and easy to watch”

hole at that time was selected Information on athletesand “Itʼs good to be able to watch in a cool room” On theand live information obtained via the Internet wasother hand there were also comments such as “I felt it

converted into content and transmitted to the 5G

was inconvenient that I could not turn the page by

networkswiping” and “How about including more detailed match

Because there were no spectators this time thedata such as “how many yards of rough on the firstservice was provided only on loaner tablets but up to

swing” and “green putt” data” and “When two or more

now the service was envisioned to be provided to thegroups are on the hole the video is not displayed” This

spectatorsʼ own 5G‑compatible devices In this casemade clear the issues that need to be improved in thevisitors must follow procedures such as downloading thefuture Regarding our goal “High‑quality images ofapplication and accessing the website to use the service

competitions can be viewed simultaneously without

Therefore having a more comfortable UI/UX (Userleaving the room” we believe that we were able toExperience) is one of the challenges In addition since

provide a new spectator experience based on the praise

spectators do not necessarily own devices with large

we received from the many people who experienced thescreens such as tablets providing content that can be

event

enjoyed on smaller screens will also be challenging

By the way although cable TV was installed in thevenue our service transmitted videos via 5G and the234

Results

The measures were implemented without significant
218

videos were delivered to the tablets faster than cable TV
because of the real‑time feature of 5G The video of the

THE JOURNAL OF THE INSTITUTE OF ELECTRONICS INFORMATION AND COMMUNICATION ENGINEERS

putt‑in in front of the lounge could be viewed in real‑other sports and there are possibilities for services

time at almost the same time as the actual competition

suited to those sports The features of 5G have been

while the video shown on the cable TV had a slightembodied in a tangible form and the practical application

delay which revealed a new feature of this service : itsof 5G is expected not only in sports but also in otherreal‑time naturefields and further development is expected

Technically there was not enough time for rehear-

Finally we would like to thank the many people whosals and we had to go through some trial‑and‑error andprovided tremendous support and cooperation in plan-

could not provide the service to individual devices

ning and implementing the TOKYO2020 5G PROJECT

which we had initially purpose to do because there were

We want to take this opportunity to express our deepest

no spectators Although there were other issues wegratitude We would also like to thank Mr Yoshiyuki

achieved excellent results at the Tokyo 2020 Olympic

Mihara of Nippon Telegraph and Telephone Corporation

Games an environment that demands a very high leveland Mr Yusuke Kobayashi of NTT DoCoMo Inc forof performance

their support in writing this article We thank you all

235

Conclusion

References

We provided a new experience of watching the Games(using tablets using commercial 5G A total of more than90 people experienced the service and it was highly
praised The communication technology used in this(project can be applied within the same competition
venue and in other competitions In the future it isexpected that 5G will be combined with other technologies and further developed into services while taking
advantage of its features such as high capacity and real‑()

Ministry of Internal Affairs and Communications, “Fifth Generation
Mobile Communications System (5G) Now and Future Prospects,”
2019.
https://www.soumu.go.jp/main_content/000633132.pdf) T. Yamaguchi, T. Sato, M. Ono, Y. Tonomura, K. Namba, Y. Kikuchi,
T. Hoshide, T. Morisumi, A. Ono, and K. Minami, “Surround Video
Stitching and Synchronous Transmission System for Immersive Live
Viewing Services,” The Journal of The Institute of Image Information
and Television Engineers, vol. 74, no. 2, pp. 402-411, 2020.
) Y. Tonomura, H. Imanaka, K. Tanaka, T. Morisumi, and K. Suzuki,“On standardization activities for Ultra High-Reality Live Experience(ILE),” ITU Journal, vol. 47, no. 5, pp. 14-17, 2017.

time performance(Received February 28 2022)

3．Review Reflection and Future Legacy
To realize one of the goals of the Tokyo 2020 Games“the most innovative Games in history that will bringpositive change to the world” we planned and implemented measures utilizing “5G”

one of the mostpromising technologies of the future The environment
surrounding the Games changed drasticallyas theGames were postponed for one year due to the effects of
the COVID‑19 and there was much discussion regarding the implementation of this project However the
project membersʼ desire to implement these measures topromote “Japan as a technological powerhouse” and theencouragement of the staff at each venue to entertain
spectators by providing new experiences prompted the

OHTO Yutaka
He received his B Eng from Kyoto University in 1987 M Eng from Kyoto University in
1989 and joined NTT in 1989 He joined NTT
docomo in 1992 and has been involved in many
projects such as visioning He is a member of
the Tokyo 2020 Organising Committee Innovation Promotion Office In 2021 he became theHead of DOCOMO CS Logistics Division
KINOSHITA Shingo
He graduated from Osaka University in 1991
with a B S in Engineering Science He
graduated from UCL the University of London
in 2007 with an M S in Management ofTechnology He joined NTT in 1991 where he
was in charge of creating new spectator
experiences for the Tokyo 2020 Games Currently as Director of NTT Human Information
Laboratories he is in charge of research anddevelopment related to information and communication processing for “humans”

resumption of discussions in the direction of implementing the measures

Unfortunately

there were “nospectators” at the venues but a sizeable number ofpeople involved in the Games experienced the projectand responded positively Although further study isneeded to make the service more attractive we believe
specific results were achieved this time Regarding a

FURUNO Noriyuki
After graduating from the University ofTokyo in 1990 and joining NTT in the same
year He later worked at NTT holdings Nishi‑
Nippon COM He worked for DOCOMO in2017 and Vice President Tokyo 2020 Business
Office In addition to the 5G project planned
and implemented measures for the Holy Fire
Relay and Athletesʼ Village

new style of spectating the project could be applied toSpecial Issue : Technology and Innovation in the Olympic and Paralympic Games Tokyo 2020219