Beyond 5G White Paper
Supplementary Volume“Sensing Technologies”

Version 1.0
March 7, 2024
Beyond 5G Promotion Consortium
White Paper Subcommittee

Preface .............................................................................................................................. 4
I. CSI-Based Device-Free Sensing Using Deep Learning with 5G NR 28 GHz Band .... 8
I-1. Introduction .............................................................................................................. 8
I-2. CSI-based device-free sensing using DNN ............................................................. 9
I-3. Experimental results ............................................................................................. 10
I-4. Conclusion .............................................................................................................. 11
II. Indoor Experimental Evaluation of Device-free Localization Schemes Using Channel
State Information in Distributed Antenna Systems ............................................. 13
II-1. Introduction .......................................................................................................... 13
II-2. CSI-based localization approaches ...................................................................... 14
II-3. Experiment scenario and results ......................................................................... 15
II-4. Conclusion ............................................................................................................. 17
III. CSI2Image: CSI-to-Image Conversion using a Generative Model .......................... 19
III-1. Introduction ......................................................................................................... 19
III-2. CSI2Image ........................................................................................................... 20
III-3. Evaluation ........................................................................................................... 21
III-4. Conclusion............................................................................................................ 22
IV. Use Cases for CSI Sensing with an Example of Pedestrian Movement Direction
Identification .......................................................................................................... 24
IV-1. Introduction ......................................................................................................... 24
IV-2. Use cases in IEEE802.11bf and commercial products...................................... 25
IV-3. Use cases in my research .................................................................................... 26
IV-4. Pedestrian movement direction identification .................................................. 26
IV-5. Conclusion ............................................................................................................ 29
V. Integrated Sensing and Communication (ISAC)....................................................... 31
V-1. Introduction ........................................................................................................... 31
V-2. ISAC use case overview ........................................................................................ 31
V-3. ISAC for centimeter-level positioning ................................................................. 32
V-4. ISAC for mm-level imaging at the THz band ..................................................... 33
V-5. Conclusion ............................................................................................................. 351

VI. Space-Time Synchronization.................................................................................... 37
VI-1. Introduction ......................................................................................................... 37
VI-2. Three technologies comprising space-time synchronization ............................ 38
VI-3. Space-Time Synchronization .............................................................................. 40
VI-4. Conclusion ............................................................................................................ 41
Abbreviation List ........................................................................................................... 422

【Revision History】
Ver.
Date
1.0 2024.3.7
Initial version

Contents3

NotePreface
In our published white paper “Beyond 5G White Paper ~Message to the 2030s~”, weresearched what are required in various industries in the Beyond 5G era, then we
proposed “Further enhancement of specific 5G features” as key features for Beyond 5G.
To address these key features, Target Key Performance Indicators (Target KPI) for
Beyond 5G have been derived, and as quantitative Target KPI related with “Sensing”,
sensing accuracy/resolution with order of centimeter (and more) has been introduced,
which is much higher than that with order of meters in 5G.
The Third Generation Partnership Project (3GPP) has initiated discussions on
Integrated Sensing and Communication (ISAC) for the secondary use of radio waves.
Defined use cases include the detection of human and animal intrusions inindoor/outdoor environments and understanding the status of automobiles andautomatic guided vehicles. Sensing technologies utilizing radio waves have foundwidespread applications in detecting the distance and direction of objects, among otherfunctionalities. With the current rapid advancements in Artificial Intelligence (AI) andMachine Learning (ML), their application scope continues to broaden, particularly inshape, motion, and gesture detections.
Sensing technologies can be considered in both aspects of Beyond 5G advancement
through sensing as well as sensing with Beyond 5G. First, let us consider the sensing for
Beyond 5G. Sensing of wireless environments in Beyond 5G wireless communications is
performed while transmitting data, and making this sensing more accurate is essential
for improving performances of the wireless communications. The sensing data obtained
here can be used not only for the wireless communications, but also for various
applications as mentioned above. Next, let us consider Beyond 5G for the sensing. The
use of high-frequency radios including millimeter-wave and terahertz bands is also
expected to realize high-precision spatial sensing and localization (including positioning)by taking advantage of their properties. Such sensing is provided by the wirelesscommunications (fixed and movable base stations) and optical communications. A basic
principle of the wireless sensing is to characterize the status and behavior of a target
object as radio propagation characteristics in wireless channels; a Received Signal
Strength Indicator (RSSI) and Channel State Information (CSI) are useful as feature
information for the wireless sensing. CSI-based sensing is crucial for integrating
communication and radar sensing. Many studies have shown its adaptability for various
sensing tasks. However, the CSI-based sensing faces an open issue: how information inphysical space (sensing environment) is reflected in CSI observations. To address this,further research is needed to better understand the relationship between the physicalspace and the CSI observations.

4In addition to such wireless sensing, network architecture to collect and process large
amounts of data from cameras, Light Detection And Ranging (LIDAR), and other sensors,
as well as sensing of real space as a digital twin are also expected.
However, there are still major challenges in the practical implementations for thedesign and evaluation of ISAC as a core technology in the Beyond 5G system. First andforemost, a theoretical framework is necessary to analyze and evaluate the performanceof current ISAC solutions to identify the benefits and any short comings. Current designof the ISAC system calls for the baseband and RF hardware to be functionally shared
and as a trade-off, the impact of distortion parameters on sensing performance needs tobe carefully considered. The challenge for the joint waveform design is the very different
KPIs for communication and sensing where optimizing both might not be so straightforward.
ISAC in the mobile communication network provides great opportunities and benefits
for synchronized multi-static sensing where the technology challenges here would lie inthe synchronization to achieve the optimum fusion sensing results. Concretely, current
Global Navigation Satellite System (GNSS) fail to provide pico-second level
synchronization accuracy to base stations, and new space-time synchronization should
be provided with that level of accuracy, enabling the phase-locked synchronization in the
millimeter wave between the base stations.
To tackle these challenges and realize new use cases, there are a lot of research anddevelopment activities on the sensing technologies in Japan. In this white paper, theseresearch and development activities and their results with a lot of figures are shown as
follows:-“CSI-Based Device-Free Sensing Using Deep Learning with 5G NR 28 GHz Band”
describes an overview of device-free sensing technology, which detects target
object without the need for mobile terminals, utilizing deep learning. It further
introduces the effectiveness of this technology by experiments using a radio
testbed equipped with the physical layer specifications of the 28 GHz-band 5G NR.

-“Indoor Experimental Evaluation of Device-free Localization Schemes Using
Channel State Information in Distributed Antenna Systems” describes a real-timechannel state information (CSI)-based device-free localization scheme for
distributed antenna systems, where CSI feedback frames are collected and used
as a dataset for machine learning (ML)-based localization. Experimental results
confirms that the localization scheme is effective for detecting a target in an indoor
environment.

5

-“CSI2Image: CSI-to-Image Conversion using a Generative Model” describes how
to convert CSI observations into RGB images corresponding to physical space
using generative adversarial network (GAN) architecture. The generated RGB
images intuitively show the relationship between the CSI observations and thephysical space, and potentially help us to extract many environmental parameters
for multi-purpose sensing system.

-“Use Cases for CSI Sensing with an Example of Pedestrian Movement Direction
Identification” describes use cases for CSI sensing from the perspectives ofcommercial products and the author’s research, and specifically the effectivenessof pedestrian movement direction identification as one of the use cases for CSI
sensing is verified by experimental evaluation with ML.

-“Integrated Sensing and Communication (ISAC)” describes a concept of ISAC,
typical use cases, and two case studies of how to use ISAC to improve localization
accuracy and perform millimeter-level imaging at the THz band using future
portable devices. The research challenges to implementing ISAC in practice are
discussed.

-“Space-Time Synchronization” describes that synchronization must not only belimited to time but also extend to space, entailing the sharing (synchronization)of spatial coordinate axes. The space-time synchronization is realized by threebasic technologies, namely compact atomic clocks, wireless time synchronization,and cluster clock systems, which are explained briefly.

In conclusion, as we embark on the journey towards Beyond 5G technologies, thesensing technologies emerge as one of key elements in this technological evolution.
Japan's endeavor to overcome the challenges to realize the sensing technologies, coupledwith its commitment to research and development in this domain, positions it at theforefront of this next-generation communication revolution. This white paper aims toprovide a comprehensive overview of the potential, challenges, and future directions of
the sensing technologies for Beyond 5G, with a particular emphasis on their initiatives
and advancements in Japan.
This White Paper was prepared with the generous support of many people who
participated in the White Paper Subcommittee. The cooperation of telecommunicationsindustry players and academia experts, as well as representatives of various industries
other than the communications industry has also been substantial. Thanks to everyone’s
participation and support, this White Paper was able to cover a lot of useful information

6

for future business creation discussions between the industry, academia, andgovernment, and for investigating solutions to social issues, not only in thetelecommunications industry, but also across all industries. We hope that this White
Paper will help Japan create a better future for society and promote significant global
activities.
Satoshi Suyama
NTT DOCOMO, INC.

7

I. CSI-Based Device-Free Sensing Using Deep Learning with 5G NR 28 GHz
Band
Tomoki Murakami, Shinya Otsuki
NTT Corporation
Yutaka Musaka, Yoshifumi Morihiro, Huiling Jiang, Satoshi Suyama
NTT DOCOMO, INC.
Yasushi Maruta
NEC corporation

Abstract— Integrated Sensing and Communication (ISAC) is gaining attraction as it
aims to bring added value to next-generation mobile communication networks. This
paper offers an overview of the device-free sensing technology, which detects target
object without the need for mobile terminals, utilizing deep learning. It further
introduces the effectiveness of this technology based on our experiments conducted on aradio testbed equipped with the physical layer specifications of the 5G (NR) 28 GHz band.
I-1. Introduction
The Japanese Cabinet Office has advocated “Society 5.0” to realize a human-centeredsociety that balances economic advancement with the resolution of social problems
through a system that highly integrates cyberspace and physical space [1]. This systemleverages artificial intelligence (AI) and machine learning (ML) to analyze vast amountsof sensor data as big data in physical space, providing feedback to humans in various
forms. Sensor data plays a pivotal role in Society 5.0 and requires efficient and costeffective integration into cyberspace. In response to this need, the Third Generation
Partnership Project (3GPP) has initiated discussions on Integrated Sensing andCommunication (ISAC) for the secondary use of radio waves. Defined use cases include
the detection of human and animal intrusions in indoor/outdoor environments andunderstanding the status of automobiles and Automatic Guided Vehicles (AGVs) [2].
Sensing technologies utilizing radio waves have found widespread applications in
detecting the distance and direction of objects, among other functionalities. With thecurrent rapid advancements in AI/ML, their application scope continues to broaden,
particularly in shape, motion, and gesture detections [3][4]. This paper specificallyfocuses on device-free sensing as a method for detecting target objects without the needfor mobile terminals. It introduces a Channel State Information (CSI)-based device-free
sensing method that achieves high-precision location detection of target objects throughthe application of a Deep Neural Network (DNN). Additionally, we demonstrate theperformance of the proposed method through the results of an indoor experiment8

conducted on a radio testbed, equipped with the physical layer specifications of the 5G(NR) 28 GHz band [5].
I-2. CSI-based device-free sensing using DNN
As outlined in the reference paper [6], the integration of DNN into radio
communication networks is advancing, with sensing technologies using radio waves
being fundamental to this progress. This paper introduces a device-free localization
method, utilizing a DNN capable of detecting the location of a target object without
relying on mobile terminals [7][8]. Fig. I-1 depicts a system model wherein the detectionof objects such as humans or cars and their states is accomplished through the analysisof radio waves between Beyond 5G (B5G) base stations (BSs). The training data for thissystem includes the target object locations and CSI between B5G-BSs, serving as thephysical information. CSI, obtained through reference signals and similar means, is vital
information for demodulation processing in radio communication networks. A prediction
model for the DNN, developed through supervised learning based on recurrent neuralnetwork architecture, is constructed using the locations of target objects and CSI.
Subsequently, the location of the target object is determined through the prediction
model and the acquired CSI. This detected location information can be stored incyberspace as sensor data, thereby contributing to the enhancement of radio
communication network quality.

Physical Info.

B5G BS

position, state,point cloud etc.

Prediction

Wireless Info.

position, state,congestion, trafficetc.

RSRP, RSRQ,
throughput, CSI etc.

Detection object

B5G BS
Fig. I-1 System model.

9• Wireless quality
improvement• Sensor (Big data)

I-3. Experimental results
We demonstrate the effectiveness of CSI-based device-free sensing utilizing a 28 GHz
radio testbed [5]. As depicted in Fig. I-2, the experimental setting comprises an indoor
office with dimensions of 25 m × 15 m × 3.5 m and four columns. The radio testbed isconfigured to meet the physical specifications of 5G NR in the 28 GHz band, employing
Multiple Input Multiple Output (MIMO) Orthogonal Frequency Division Multiplexing(OFDM) transmission with a 100 MHz bandwidth and a subcarrier spacing of 60 kHz.
To enhance sensing accuracy, antennas for each BS are strategically distributed, as
illustrated in Fig. I-2. The CSI for 6 × 2 MIMO-OFDM is acquired at 1-millisecond
intervals through reference signals exchanged between BSs. The target object in thisexperiment is a human body phantom adjusted to match the dielectric constant of an
actual human body in the 28 GHz band. Moreover, the phantom is mounted on an AGV
for automated movement within the designated area, facilitating precise location
information acquisition.
Fig. I-3 shows the cumulative distribution of the distance error between the detected
location obtained from the proposed DNN-based method and the actual location. The
predictive DNN model incorporates pre-acquired location information of the human body
phantom and CSI from the BS. This figure highlights that the median location error forthe human body phantom is approximately 0.6 m, with a root mean squared error(RMSE) of 1.1 m. While previous studies have demonstrated the effectiveness of location
detection using Wi-Fi within sub-6 GHz bands, these results affirm the feasibility oflocation detection in the 28 GHz band.
25m

Object
1

1 2

3

4

2 5

6

BS antenna

human body
phantom

14m

AGV

No.

Antenna location at BS-1

No.

Antenna location at BS-2
Moving area of object

Fig. I-2. Experimental environment.

10

RMSE 1.1m

Approx. 1.4m (80%value)

Approx. 0.6m (50%value)
Approx. 0.4m (20%value)

Fig. I-3. Experimental results.
I-4. Conclusion
This paper outlines the CSI-based device-free sensing method for detecting a target
object without the need for mobile terminals. Furthermore, we demonstrated thefeasibility of achieving 1-meter-class localization through indoor experiments conducted
on a radio testbed equipped with the 5G (NR) 28 GHz band. In the future, we intend tovalidate its application in outdoor scenarios and explore the detection capabilities for
multiple objects.
REFERENCE[1] Cabinet office, “Society 5.0”, https://www8.cao.go.jp/cstp/english/society5_0/index.html[2] 3GPP TR 22.837, “Feasibility Study on Integrated Sensing and Communication”[3] M. Yongsen, G. Zhou, and S. Wang, “WiFi sensing with channel state information: A
survey,” ACM Computing Surveys, vol.52, no.46, June 2019.
[4] J. Liu, H. Liu, Y. Chen, Y. Wang, and C. Wang, “Wireless sensing for human activity:
A survey,” IEEE Communications Surveys & Tutorials, vol.22, no.3, Aug. 2019.
[5] NTT, NTT DOCOMO, NEC, “World's first successful demonstration of distributed
MIMO that continues wireless connections in the 28 GHz band by eliminating shielding
issues,” https://group.ntt/en/newsrelease/2022/10/31/221031a.html[6] C. Zhang, P. Patras, and H. Haddadi, “Deep learning in mobile and wirelessnetworking: survey,” IEEE Communications surveys & tutorials, vol.21, no.3, March 2019.

11

[7] R. Kudo, K. Takahashi, N. Nagata, T. Murakami, T. Ogawa, and K. Takasugi,“Wireless Link Quality Prediction Using Physical Space Information based on Deep
Learning,” IEICE transaction, J105-B, no. 10, Oct. 2022 (Japanese).
[8] R. Kudo, K. Takahashi, T. Murakami, H. Yoshioka, and T. Ogawa, “RNN Position
Estimation using a Single WLAN AP and Performance Evaluation,” Proc. the 2021 IEICE
Society Conference, B-15-41, 2021 (Japanese).

12II. Indoor Experimental Evaluation of Device-free Localization Schemes
Using Channel State Information in Distributed Antenna Systems
Osamu Muta
Kyushu University
Tomoki Murakami, Shinya Otsuki
NTT Corporation

Abstract—Wireless communication system-based localization techniques that use
channel state information (CSI) have attracted much attention. Performance of the CSIbased localization schemes depends strongly on the selected feature information andantenna placement. Herein, we present a real-time CSI-based device-free localization
scheme for distributed antenna systems, where CSI feedback frames are collected andused as a dataset for machine learning (ML)-based localization. Experimental results
confirmed that the developed localization scheme is effective for detecting a target in anindoor environment. We also discuss how much performance improvement can be
expected when antenna placement is given properly.
II-1. Introduction
Wireless sensing is a key technology supporting the evolution of wirelesscommunication for Beyond-5G and 6G networks [1]. The basic principle of wirelesssensing is to characterize the status and behavior of a target object as radio propagation
characteristics in wireless channels; a received signal strength indicator (RSSI) andchannel state information (CSI) are useful as feature information for wireless sensing.
Recently, indoor device-free object detection approaches using radio signals of existing
wireless communication infrastructure, have been investigated [2]-[7], where large
amounts of CSI in the frequency and spatial domains are acquired simultaneously using
multi-input multi-output (MIMO) transmission with orthogonal frequency divisionmultiplexing (OFDM). In [5]-[7], IEEE802.11ac-based wireless local area network(WLAN)-based device-free indoor object detection schemes were proposed, by which CSI
feedback frames in WLANs were collected and analyzed to detect a target object and its
behavior. However, the performance achieved by the CSI-based approaches depends
strongly on the antenna placement and the surrounding propagation environment.
This paper introduces our recent studies of wireless communication system-basedindoor device-free localization where feedback beamforming weights are used as effective
feature data for machine learning (ML)-based object detection and localization.
Experiment results demonstrate that our developed algorithm works well with small
datasets in an indoor environment when distributed antenna placement is accomplishedproperly.

13

II-2. CSI-based localization approaches
Fig. II-1 presents an illustration of the principles of device-free localization in wirelesscommunication systems where the CSI between the base station (BS) and the terminal
is affected by the target presence. Consequently, it is expected to detect the target bylearning the relation between the states of the target and of the wireless channel without
requiring that the target have wireless devices. However, a challenging hurdle isacquisition of a sufficient amount of CSI from environments. An effective method is to
use CSI in existing WLAN systems [5].

Terminal
Base station

Amplitude

Frequency-domain CSI
with object
w/o object

TargetFrequency

Fig. II-1. Principles of device-free localization using CSI.
MIMO channel
Target

S

FFT
• • •

• • •

• • •

• • •

Base
stationTerminalN
Channel estimationand demultiplexing

FFT
Feedback
to AP

CSI
Compression

SVD

CSI capturing terminal
Captured CSICSIConcatenation•

MachineLearning

Fig. II-2. Block diagram of the device-free object detection system.
Fig. II-2 portrays a block diagram of the object detection and localization system
consisting of an BS, a terminal, and a CSI-capturing terminal, where M, N, and S
respectively denote the numbers of transmit antennas, received antennas, and streams.
On the terminal side, after channel estimation, a compressed version of a right-singularmatrix obtained by singular value decomposition of the channel matrix is fed back to the
BS side as beam-forming weights (BFWs). After the CSI-capturing terminal collects
feedback frames sent by the terminal, it extracts the compressed CSI samples and uses14

them for ML-based localization [5]-[7]. To improve the localization performance, wedeveloped an effective lightweight algorithm with a small dataset [6], where current andpast BFWs are concatenated as single data to build more accurate feature data. In
addition, a frequency-domain sampling-based CSI compression [6] is adopted tominimize the dataset and the required complexity. After applying frequency-domain
sampling and concatenating multiple BFWs as single feature data, they are used for bothoff-line training and for on-line detection.

Fig. II-3. Experiment scenario and setup.

II-3. Experiment scenario and results
To clarify the effectiveness of the developed algorithm in a real environment, we
conducted experimental evaluations of indoor localization with the developed algorithm.
The experiment scenario and setup using IEEE802.11ac-based WLAN [8] are depicted
in Fig. II-3. Details of the experiments are presented in an earlier report of the relevant
literature [7]. The detection area is divided into R regions labeled as 1, …, R=32. For thisexperiment, we consider a multi-class classification problem to detect the location (label)of a single target object. Random Forest model is used and built by off-line training withmeasured CSI. We evaluate the detection probability, which is defined as a conditional
probability that the ML result is the same as the actual label number where a target

15

person is located in one of the R=32 labeled areas. In this scenario, the detection
probability is evaluated when a few (Ms=4) antennas are selected among numerous
distributed antennas (M=12) to elucidate the relation between antenna placement andthe detection probability.
Fig. II-4 (a) presents the detection probabilities for all possible antenna patterns
where Ms=4 antennas are selected among M=12. The total number of antenna
combinations is 495. The horizontal axis index shows the antenna pattern numbers
sorted from left to right in descending order of the average detection probability. The
result indicates that about 20-point performance improvement can be confirmed for theantenna patterns with the maximum (best) and minimum (worst) detection probabilities.
Heatmaps of the best 5 and the worst 5 area-wise detection probabilities are also shown
respectively in Fig. II-4 (b) and Fig. II-4 (c). Results imply that a lower detection
probability (better performance) tends to be obtained when the AP antenna positions are
distributed.

Average detection probability

1

0.9

0.8

(b) The best 5 detection probability (heatmap)
0.7

0.6
1

100

200

300

495

400

Antenna pattern number sorted in descending order of(a) Average detection probability for different antenna positions(c) The worst 5 detection probability (heatmap)

Average detection probability

Fig. II-4. Average detection probability for selected antenna positions.
1
Selected antenna numbers in Fig.3
A (5, 6)

0.8

0.6

B

(5, 6, 7, 8)

C(1, 2, 5, 6, 7, 8)

D (1, 2, 3, 4, 5, 6, 7, 8)

0.4
0.2

E

(1, 2, 3, 4, 5, 6, 7, 8, 11, 12)

F(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)

G (3, 5)
H (3, 5, 6, 12)

0
2

10
4
6
8
Number of selected antennas
#A~F

#G~K

12

Best caseI(1, 3, 5, 7, 10, 12)

J(1, 3, 4, 5, 6, 7, 10, 12)

K (1, 3, 4, 5, 6, 7, 8, 10, 11, 12)

Fig. II-5 Average detection probability in case of Ms = 2, 4, 6, 8, 10, and 12.

16

Fig. II-5 portrays the detection probability for a particular antenna placement interms of the number of selected antennas. The table in the figure presents the selected
antenna elements for each case, which corresponds to the antenna numbers in Fig. II-3(a). This figure shows that distributing the antenna placement improves the localization
performance, which consequently maximizes the average detection probability. Results
indicate that the overall detection probability improves as Ms increases because Ms
becomes greater, and because more features are used for ML-based localization. Thisfinding implies that, if using the optimal antenna pattern for Ms=4 is possible, thenaverage detection probability comparable to the case of Ms=12 can be achieved.
II-4. Conclusion
As described herein, we introduced a device-free localization scheme using
concatenated CSI in wireless communication systems with distributed antennas.
Experiment results demonstrate that the developed scheme works well with small
datasets in an indoor environment scenario. Moreover, we have demonstrated that theantenna placement strongly affects the achievable localization performance in an indoor
environment. Developing object detection algorithms with a more powerful ML model isleft as a subject for our future work.
Acknowledgments
We thank Mr. Junsuke Izumi and Mr. Shunsuke Shimizu, master course students at
Kyushu University, for their contributions to this research.
REFERENCE[1] “White Paper 5G Evolution and 6G (Version 5),” NTT DOCOMO, Inc., Jan. 2023,
https://www.docomo.ne.jp/english/binary/pdf/corporate/technology/whitepaper_6g/DOCO
MO_6G_White_PaperEN_v5.0.pdf[2] A. Khalajmehrabadi, N. Gatsis, and D. Akopian, “Modern WLAN Fingerprinting
Indoor Positioning Methods and Deployment Challenges,” IEEE Communications
Surveys & Tutorials, vol. 19, no. 3, pp. 1974–2002, Oct. 2017.
[3] H. Jiang, C. Cai, X. Ma, Y. Yang, and J. Liu, “Smart Home Based on WiFi Sensing: A
Survey,” IEEE Access, vol. 6, pp. 13317–13325, March 2018.
[4] H. Zhu, F. Xiao, L. Sun, R. Wang, and P. Yang, “R-TTWD: Robust Device-free Throughthe-wall Detection of Moving Human with WiFi,” IEEE Journal on Selected Areas inCommunications, vol. 35, no. 5, pp. 1090-1103, March 2017.
[5] T. Murakami, M. Miyazaki, M. Ishida, and A. Fukuda, “Wireless LAN Based CSI
Monitoring System for Object Detection,” MDPI Electronics, Nov. 2018.

17

[6] O. Muta, K. Takata, K. Noguchi, T. Murakami, and S. Otsuki, “Device-Free WLAN
Based Indoor Localization Scheme with Spatially Concatenated CSI and Distributed
Antennas,” IEEE Trans. Vehicular Technology, vol. 72, no. 1, pp. 852-865, Jan. 2023.
[7] S. Shimizu, O. Muta, K. Noguchi, T. Murakami, and S. Otsuki, “Performance of
WLAN-based Object Detection with Distributed Antenna and Spatially Concatenated
CSI,” Proc. IEEE VTC-Fall 2023 Workshop, Oct. 2023.
[8] IEEE Computer Society, IEEE 802.11-2016, LAN/MAN Standards Committee, 2016.

18

III. CSI2Image: CSI-to-Image Conversion using a Generative Model
Sorachi Kato, Takuya Fujihashi, Takashi Watanabe, Shunsuke Saruwatari
Osaka University
Tomoki Murakami
NTT Corporation

Abstract—Wireless sensing studies based on channel state information (CSI) continue
to be successful in various sensing tasks. However, we still have no clear answer to what
extent we can extract the environmental parameters of physical space from CSI. We
proposed CSI2Image to address such a challenging issue. It converts CSI observations
into RGB images corresponding to the physical space using generative adversarial
network (GAN) architecture. The generated RGB images intuitively show the
relationship between the CSI observations and the physical space, and potentially help
us to extract many environmental parameters for multi-purpose sensing system.
III-1. Introduction
Wireless sensing is becoming an increasingly attractive sensing technique in Beyond
5G and 6G networks due to its potential to solve issues associated with conventional
sensor-based sensing and its ability to provide extensive, precise, and non-invasive
sensing. CSI based sensing is crucial for integrating communication and radar sensing.
Many studies have shown its adaptability for various sensing tasks, including activity
recognition [1], [2], vital signal sensing [3], [4], and localization [5], [6]. However, CSI
based sensing faces an open issue: how information in physical space is reflected in CSI
observations. To address this, further research is needed to better understand therelationship between physical space and CSI observations.
To tackle the challenging issue, this paper introduces CSI2Image [7]. It converts CSI
observations in a more intuitive format, RGB images, allowing us to comprehensively
understand how CSI reflects the physical space. CSI2Image generates images of thephysical space from CSI observations using GAN architecture in an end-to-end manner.
Experiments demonstrated that the well-trained CSI2Image can generate the snapshotsof the physical space from the CSI observations by extracting rich information from them.
In addition, thanks to the advancement of sophisticated image recognition techniques,
we can extract the various properties of the physical space using the generated images
and image recognition techniques.

19III-2. CSI2Image
Fig. III-1 shows the overview of the proposed CSI2Image. The basic architecture isbased on DCGAN [8], thus the models are trained in an adversarial manner: thegenerator aims to generate realistic images, while the discriminator tries to distinguishbetween real and generated images. The generator takes both CSI observations andlatent variables from a standard normal distribution as input. In addition, it is assumed
that we have a pair of time-synchronized CSI observations and images of the targeted
physical space.
We experimentally confirmed that simply introducing DCGAN does not lead tosatisfactory results. We proposed a unique training loop called “Hybrid Learning”, whichaims to ensure the generalizability of the networks for both image generation andreal/fake determination, as well as obtaining the precise mapping from CSIs to theimages. Fig. III-2 shows the three different training steps included in the Hybrid
Learning. The first step is CSI2Image Learning, where the generator is supervised
trained using CSI observation and their corresponding images, followed byDiscriminator Learning, where the discriminator is trained with real images andgenerated images, but this time the generator generates images from random latentvariables. Finally, after every K iteration, Generality Learning is triggered to update theparameters of networks and deceive the discriminator with the images generated by CSI
observations. The generator acquires the mapping from the probability field of CSI tothat of the image domain while avoiding overfitting to the training dataset through the
combination of direct supervision and adversarial training. The training loop
successfully displays the surrounding environment in the physical space containing in
CSI observations in RGB images. To extract the properties of the physical space fromthe images, we can employ existing vision-based object detection techniques like YOLO
[9] according to the task specification.

Fig. III-1. The overview of CSI2Image.

20

(a) CSI2Image Learning(b) Discriminator Learning(c) Generality Learning
Fig. III-2. The proposed hybrid learning.
III-3. Evaluation
To demonstrate the performance of CSI2Image to accurately generate RGB images ofan indoor space from CSI observations and its performance in different sensing tasks,
we conducted two experiments: object classification and human location classification.
As shown in Fig. III-3, CSI2Image successfully generated the RGB images and achieved
over 90% accuracy for both tasks. Especially for human location classification task, we
compared the efficiency of our proposed Hybrid Learning compared to supervised
learning only with the generator (gonly) and original DCGAN training scheme (gan).
Experimental results show that Hybrid Learning outperforms the other two methods interms of detection rate, similarity score between generated images and ground truth,bounding box confidence scores, and classification accuracy. This advantage is even more
pronounced in more complex scenarios involving two people.

21

Fig. III-3. Qualitative and quantitative evaluation for different sensing tasks.
(top) Object classification, (bottom) Human location classification.
III-4. Conclusion
This paper introduced the overview of CSI2Image, a GAN-based CSI-to-image
conversion method. We demonstrated that the combination of direct supervision andadversarial training successfully achieves the conversion. The conversion not only helps
us to represent physical information CSI has in a more explicable format but also helps
us to construct multi-purpose sensing system with the vision-based detection techniques.
Acknowledgements
This work was supported in part by the Japan Society for the Promotion of Science(JSPS) KAKENHI under Grant JP19H01101 and Grant JP17KT0042, and in part by the
JST PRESTO, Japan, under Grant JPMJPR2032.
REFERENCE[1] M. Muaaz et al., “Wi-Sense: a passive human activity recognition system using Wi-Fi
and convolutional neural network and its integration in health information systems,”

Ann. Telecommun., vol. 77, no. 3, pp. 163–175, Apr. 2022.
[2] W. Meng et al., “GrapHAR: A Lightweight Human Activity Recognition Model by
Exploring the Sub-carrier Correlations,” IEEE Trans. Wireless Commun., pp. 1–1, 2023.
[3] Z. Guo et al., “BreatheBand: A Fine-grained and Robust Respiration Monitor System
Using WiFi Signals,” ACM Trans. Sen. Netw., vol. 19, no. 4, pp. 1–18, May 2023.
[4] Y. Li et al., “DiverSense: Maximizing Wi-Fi Sensing Range Leveraging Signal
Diversity,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 6, no. 2, pp. 1–
28, Jul. 2022.

22

[5] X. Zhang et al., “AutoLoc: Toward Ubiquitous AoA-Based Indoor Localization Using
Commodity WiFi,” IEEE Trans. Veh. Technol., vol. 72, no. 6, pp. 8049–8060, Jun. 2023.
[6] Z. Wang et al., “Single-target real-time passive WiFi tracking,” IEEE Trans. Mob.

Comput., vol. 22, no. 6, pp. 3724–3742, Jun. 2023.
[7] S. Kato et al., “CSI2Image: Image Reconstruction From Channel State Information
Using Generative Adversarial Networks,” IEEE Access, vol. 9, pp. 47154–47168, 2021.
[8] A. Radford et al., “Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks,” arXiv [cs.LG], Nov. 19, 2015.
[9] J. Redmon et al., “You only look once: Unified, real-time object detection,” CVPR, 2016,
pp. 779–788.

23

IV. Use Cases for CSI Sensing with an Example of Pedestrian Movement
Direction Identification
Masakatsu Ogawa
Sophia University

Abstract—The original purpose of communication is to convey information. Channel
state information (CSI) is used for high-speed transmission and can also function as asensor. Adding sensing ability to the communication function is expected to open up newservices and applications. This paper describes use cases for CSI sensing from theperspectives of commercial products and my research, specifically pedestrian movement
direction identification.
IV-1. Introduction
Wireless communication has traditionally been used to carry information throughspace as a medium. Most wireless communication systems currently use MIMO-OFDM
transmission to achieve high-speed transmission rates. This method uses channel state
information (CSI) that indicates the radio propagation condition between thetransmitter and receiver. CSI can also be used as a sensor since it indicates the spatial
information between a transmitter and receiver. The sensing using this information
corresponds to incorporating sensing into communication, which has been discussed in"Beyond 5G and 6G" and "IEEE802.11 wireless LAN". The technical terms for this areintegrated sensing and communication (ISAC) and joint sensing and communication(JSAC). However, most current wireless communication chipsets do not provide aninterface for users to obtain CSI.
While research activity for CSI sensing is currently high, the number of commercial
products for Wi-Fi sensing is rare. In my opinion, the reason is that the service
requirements of use cases are strict, and the requirements are not entirely satisfied dueto the uncertainty of radio propagation. Compared with RSSI, which also indicates theradio propagation condition, CSI has a higher reproducibility; thus, the use of CSI issuitable for sensing. The disadvantage of CSI is that it depends on the transmitter’s andreceiver’s location. When the location changes, the CSI also differs from the change
before its location.
IEEE802.11bf discusses use cases for sensing [1] but does not confirm the feasibility.
The paper describes use cases of CSI sensing applications from commercial products andmy research, including advantages and disadvantages from the feasibility viewpoint.

24

IV-2. Use cases in IEEE802.11bf and commercial products
The difference between mobile communication and wireless LAN is the coverage area.
The use cases differ depending on whether the sensing target exists in an outdoor public
space or an indoor room. CSI indicates the radio propagation condition between thetransmitter and receiver. Therefore, detecting a specific sensing target is difficult in an
outdoor public space because of the wide area.
IEEE802.11bf discusses use cases for existing wireless LAN standards, which operate
at 2.4GHz, 5GHz, 6GHz, and 60GHz [1]. The range resolution relates to the signal
bandwidth, i.e., the frequency band. The specific use case for the high-frequency band,i.e., 60GHz, is high-resolution sensing, e.g., gesture recognition (hand or figure
movement), and for the low-frequency band, is low-resolution sensing, e.g., humanpresence and motion detection [2]. Among them, CSI sensing mainly corresponds to lowresolution sensing. The use cases discussed in IEEE802.11bf are as follows: Roomsensing, Gesture recognition for full-body movement, Health care, and Car sensing.
I found some information about the feasibility of the above use cases. Hex Home by
Origin Wireless is a commercial product for room sensing and healthcare [3]. Specifically,
room sensing is used for home security by detecting intruders and is also for homemonitoring of older people and/or children. The CSI variation is used to detect human
movement. Healthcare is used for measuring breathing rate using the CSI periodicity.
Wiz product named SpaceSense by Signify is another commercial product that
corresponds to room sensing, i.e., a smart light [4]. This light control uses the CSI
variation relating to the detection of human movement. Although I could not confirm
whether car sensing by Murata Manufacturing has been commercialized, CSI sensing isused to detect the presence of a child in the car by measuring movement detection andbreathing rate [5]. Note that the accuracy of breathing rate measurement depends on
the location of the transmitter and receiver.
Fig. IV-1 shows the relationship between use cases and the purpose of the product.
CSI variation is used for presence detection, and there are many examples of its
application. Combining presence detection and breathing rate measurement for intruder
detection is possible, but I could not find such a product.

25

Fig. IV-1. Relationship between use cases and the purpose of products.
IV-3. Use cases in my research
My research aims to investigate the potential of CSI sensing without considering theuser’s needs. Basically, the commercial products mentioned above do not require
machine learning. Based on my experience with Wi-Fi, use cases for CSI sensing can be
categorized into those that require machine learning and those that do not.
CSI sensing can be used for various applications without machine learning. Here aresome examples of CSI sensing use cases that do not require machine learning: breathing
rate measurement [6], people counting using breathing rate measurement [6], andpropeller rotation speed measurement [7].
CSI sensing can also be used with machine learning for more advanced applications
such as human activity recognition [8], material identification [9], pedestrian movement
direction identification [10], human location estimation [11], water height estimation ina bottle [12], pose estimation [13], and laundry dryness estimation [14].
IV-4. Pedestrian movement direction identification
Pedestrian movement direction identification is one of the use cases for CSI sensing.
Nowadays, access points are often installed on the ceilings of offices and are connected
to Ethernet or wireless mesh networks. In the future, if it becomes possible to measure
CSI from communication between access points, it will be possible to estimate humanflow.
Suppose consider a crossroads in a hallway. As shown in Fig. IV-2, one transmitterand three receivers are installed. Due to cost constraints, it is necessary to reduce thenumber of receivers as much as possible. There are 13 types of movement directions: four
entrances to the crossroads and three exits from the crossroads per entrance. In addition
to the twelve conditions, there is the condition that no humans are at the crossroads.
Three people walked ten times for CSI measurements for each direction, resulting in 30
samples per direction. Including the case where no humans are at the crossroads, thetotal number of samples is 390. If the CSI differs in every direction, it is necessary to pay

26attention to the time series of CSI. Therefore, LSTM is used as a machine learningalgorithm to classify the 13 types of movement directions, and accuracy is evaluated byleave-one-out cross-validation.

E

RxE
RxS

S RxE

N

RxN

Tx

RxN

RxS
TX

W→E

W

N→S

2

Accuracy

Fig. IV-2. Experimental environment.
1
0.98
0.96
0.94
0.92
0.9
0.88
0.86
0.84
0.82
0.8

Single receiver

Two receivers

Threereceivers

Receivers
Fig. IV-3. Experimental result.
The accuracy of the difference in the number of receivers is shown in Fig. IV-3. The
accuracy improves with the number of receivers and is over 90% when the number ofreceivers is two or three. When the receiver is only RxE, the accuracy is less than 90%.
Fig. IV-4 shows the time-series CSI in three receives. When a human walks from N to
S, the CSI fluctuation at only RxE is short. Because of the short time of the CSI
fluctuation, it isn't easy to distinguish when a human walks from N to S and when ahuman walks from S to N. The confusion matrix using only RxE is shown in Fig. IV-5,
where the label indicates direction and "nothing" indicates no human exists. I found thatthe estimation error occurs between StoN and NtoS. Therefore, the receiver at a nonline-of-sight location from the transmitter needs to be set to achieve higher accuracy.

27

E

RxE
Movingfrom N to S

N

RxS
S

RxN
TX

W

Fig. IV-4. CSI at each receiver.

Fig. IV-5. Confusion matrix at RxE.

28

IV-5. Conclusion
The paper describes use cases for CSI sensing from commercial products and myresearch, specifically pedestrian movement direction identification. Although only somecommercial products utilize CSI sensing, various services are expected to be created in
the future by utilizing this technology.
REFERENCE[1] Assaf Kasher, et al, "WiFi Sensing Uses Cases," doc.: IEEE 802.11-20/1712r2, January
2021.
[2] Claudio da Silva, et al., "Wi-Fi sensing: Usages, requirements, technical feasibility andstandards gaps," doc.: IEEE 802.11-19/1293r0, July 2019.
[3] Origin Wireless, Hex home.
https://myhexhome.com/[4] Signify, Wiz.
https://www.wizconnected.com/en-us/explore-wiz/spacesense[5] Murata Manufacturing, Wi-Fi sensing for child presence detection.
https://solution.murata.com/en-global/technology/child-presence-detection[6] R. Tokumine, M. Ogawa, "Human Counting by Estimating Breathing Rate using WiFi CSI," Proc. IEICE Tech. Rep., vol. 122, no. 341, SeMI2022-78, pp. 30-33, Jan. 2023
(Japanese).
[7] M. Ogawa, T. Takahashi, "Propeller’s Rotational Speed Measurement by WLAN
sensing," IEEJ Trans. on Industry Applications, vol. 142, no. 12, pp. 941-947, Dec. 2022
(Japanese).
[8] M. Ogawa, H. Munetomo, "Wi-Fi CSI-Based Outdoor Human Flow Prediction Using a
Support Vector Machine," Sensors, vol. 20, no. 7, 2141, 2020.
[9] Y. Tian, M. Ogawa, "Material Identification of Moving Objects by Wi-Fi Sensing Using
Machine Learning," Proc. of ICETC 2021, Dec. 2021.
[10] K. Isono, M. Ogawa, "CSI-based Crossroads Movement Direction Identification using
Ceiling-mounted APs," Proc. the 2022 IEICE General Conference, B-15-28. 2022
(Japanese).
[11] S. Usaka, M. Ogawa, "Comparison of Position Estimation Accuracy in Polar andRectangular Coordinate Systems Using Wi-Fi CSI," Proc. of APWCS 2023, Aug. 2023.
[12] S. Nishino, M. Ogawa, "Water Height Estimation by Wi-Fi CSI independent of Bottle
Size," Proc. IEICE Tech. Rep., vol. 122, no. 341, SeMI2022-85, pp. 57-60, Jan. 2023
(Japanese).
[13] Y. Yamahata, M. Ogawa, "Human Pose Estimation by Wi-fi CSI using RGB-D
Camera," Proc. the 2022 IEICE General Conference, B-15-26. 2022 (Japanese).

29[14] M. Ogawa, "A Study on Dry Estimation of Laundry by Wi-Fi Sensing," Proc. the 2023
IEICE Society Conference, B-15-5, 2023 (Japanese).

30

V. Integrated Sensing and Communication (ISAC)
Chen Yan, Huawei Technologies
Koshimizu Takashi, Huawei Technologies Japan

Abstract—6G will serve as a distributed neural network for the future Intelligence ofEverything. Network Sensing and Native AI will become two new usage scenarios in theera of connected intelligence. 6G will integrate sensing with communication in a single
system. Radio waves can be exploited to "see" the physical world and make a digital twinin the cyber world. This paper introduces the concept of integrated sensing andcommunication (ISAC) and typical use cases, and provides two case studies of how to use
6G ISAC to improve localization accuracy and perform millimeter level imaging using
future portable devices. The research challenges to implementing ISAC in practice are
discussed.
V-1. Introduction
In 6G mobile communication systems, the use of higher frequency bands (frommmWave up to THz), wider bandwidth, and massive antenna arrays will enable highaccuracy and high-resolution sensing, which can help implement the integration of
wireless signal sensing and communication (ISAC)[1] in a single system for their mutual
benefit. On the one hand, the entire communications network can serve as a sensor. The
radio signals transmitted and received by network elements and the radio wave
transmissions, reflections, and scattering can be used to sense and better understandthe physical world. The capabilities to obtain range, velocity, and angle information fromthe radio signals can provide a broad range of new services, such as high accuracy
localization, gesture capturing and activity recognition, passive object detection andtracking, as well as imaging and environment reconstruction [2]. This is called "network
as a sensor". On the other hand, the capabilities of high-accuracy localization, imaging,and environment reconstruction obtained from sensing can improve communication
performance.
V-2. ISAC use case overview
Wireless sensing has long been a separate technology developed in parallel with themobile communication systems. Positioning is the only sensing service that mobile
communication systems (until 5G) could offer. General sensing rather than positioning
will become a new function integrated into the 6G mobile communication system. This
capability will open up brand new services for 6G. These services are currently provided
by various dedicated sensing equipment, such as radar, light detection and ranging(LIDAR), and professional CT and MRI equipment. The ISAC capability will thus enable31

many new services that mobile communication system operators can offer. These include
very high accuracy positioning, localization and tracking, imaging for biomedical andsecurity applications, simultaneous localization and mapping to automatically constructmaps of complex indoor or outdoor environments, pollution or natural disastermonitoring, gesture and activity recognition, flaw and material detection and many otherservices. These services will in turn enable application scenarios in all kinds of businessfor future consumers and vertical industries. The potential new services that could be
supported by future ISAC systems are listed in Table 1. In the table, the use cases are
categorized into four functional categories across different applications/industries(vertical industry, consumer and public services):
 High-accuracy localization and tracking
 Simultaneous imaging, mapping and localization
 Augmented human sensing
 Gesture and activity recognition

V-3. ISAC for centimeter-level positioning
V-3.1. Background, motivation, and high-level scheme
The integration of sensing and communication functions can happen at three different
levels, from loosely coupled to fully integrated. At the lowest integration level, sensing
and communication capabilities can co-exist on hardware by sharing the spectrum,
which is more efficient than dedicated spectrum usage. Sensing can benefit from theeconomies of scale in the mobile communication network, where shared hardware will
be cost effective and eases deployment and 6G requires solutions for sub-centimeter level
positioning techniques for various future applications and use cases. This level ofaccuracy for positioning requires much more detailed knowledge of the radio signal
propagation environment where sensing comes into play. By learning the environment
RF map and the way the transmitted waveform is manipulated by it, the UE position
can be obtained as a function of the measurement parameters. This way, the multipath
nature of the propagation channel will be helpful. Moving to higher frequencies canfurther facilitate such sensing-assisted positioning because the channel becomes sparser,
and hence, characterizing the mapping between UE position and its propagation channel
takes less effort. In a reflection-dominant environment (which is the case in higher
frequencies), one such mapping can be obtained by decomposing the multipath channel
as multiple LOS channels coming from multiple anchors. Those anchors are obtained bymirroring the transmission point (TP) over the surface of the corresponding reflector foreach path. Those virtual anchors are referred to as virtual TPs or vTPs.

32

Fig. V-1. Mapping the objects/reflectors of the environment to virtual anchors, i.e.,
mapping multipath components to vTPs.
V-4. ISAC for mm-level imaging at the THz bandTHz lies between the mm-Wave and infrared frequencies, and thus has millimeterlevel and even sub-mm level wavelength, making the ISAC system at the THz band
(ISAC-THz) particularly suitable for high resolution sensing applications such as
millimeter-level resolution 3D imaging. Like the other lower frequency radio waves, THz
can penetrate some obstacles, achieving high-precision sensing in all weather andlighting conditions. Recent developments in semiconductor technology have bridged the"THz band gap" and made the hardware feasible at the terminal side. ISAC-THz based
portable devices will thus open the door for numerous new sensing applications such as
augmented human sensing with very high resolution.
V-4.1. Compressed sensing-based tomography imaging
A major challenge for the virtual aperture imaging technique is the irregular scanning
trajectory caused by the user moving the ISAC imaging module to perform THz scanningon an object. Assume a zigzag scanning routine is used to image an object, as shown in
A major challenge for the virtual aperture imaging technique is the irregular scanning
trajectory caused by the user moving the ISAC imaging module to perform THz scanningon an object. Assume a zigzag scanning routine is used to image an object, as shown in
Fig. V-2. The echo samplings in the horizontal direction are continuous, i.e., the spatial
spacing between sampling points is comparable to the wavelength of the echo signal.
However, continuous sampling cannot be maintained in the vertical direction. As a result,the echo samplings in the vertical direction are sparse, which will cause high and nonuniform sidelobe effects, giving rise to false artifacts, which may lead to imaging failure.
To solve this challenge, we consider decomposing the scanning trajectory on a twodimensional (2D) plane into several sets of linear scanning tracks along the horizontal

33

direction, where the sparseness of the sampling signals in the vertical domain is thenequivalent to the sparseness between horizontal tracks, as illustrated in Fig. V-2. In
this case, the reflected/echo information from the object can be retrieved from these
vertically sparse samplings via compressed sensing techniques [3]. As depicted in Fig.
V-3 (a), the robotic arm scans at a speed of 1 m/s with the scanning area set as 10 cm by
12 cm in the prototype.

Fig. V-2. Illustration of the sparse scanning approach and the tomographic imaging
techniques.

Fig. V-3. Setup of the ISAC-THz prototype.
The longitudinal spacings of the scan trajectories are controlled to simulate thesparsity in the trajectories of the user's hand-held scanning behavior. The target object
to be imaged, as shown in Fig. V-3 (b), is put in a box with a cap on top of it. As we cansee from Fig. V-3 (b), the smallest distance in the hallowed pattern is 3.5 mm, so thehighest resolution of the imaging results can be 3.5 mm. The proof-of-concept THz

34

imaging performances with different sparsity configurations in the scanning patterns
are presented and compared in Fig. V-4. In each of the figures, the 3D imaging results
are shown on the left and the cross-range profile perceived from top down is shown on
the right. The non-sparse full aperture scanning in Fig. V-4 (a) is an ideal case, in which
the vertical sampling is half wavelength adjacent. This achieves the best PSLR and ISLR
performance, which is set as an upper bound performance reference. Then, in order tosimulate the sparsity in real free hand scanning, we assume different sparsity
configurations in tests, from 50% (medium sparsity) to 25% (most sparsity), where X%
sparsity means that there are X % of the full samplings remaining in the verticaldirection. With the collection of fewer samplings, stronger sidelobe interference occurs
at the resulted aperture, resulting in worse imaging performance. From the comparisonof Fig. V-4 (c) and Fig. V-4 (d), we see that when the sparsity is too high, the traditional
tomography algorithm is not enough to recover the images. In this case, the compressedsensing based tomography approach showed its superior performance.

Fig. V-4. Imaging results at different sparsity configurations.
V-5. Conclusion
With the concept of ISAC being commonly accepted as one of the key technology trends
for 6G, this paper takes a step forward and elaborates two case studies on how 6G ISAC
technologies can be applied to improve localization and to perform high resolution
imaging. In particular, the proposed SAPE scheme utilizes the joint benefit of device freeand device-based sensing and greatly improves the positioning accuracy compared withthe current NR scheme. The prototype of the THz camera justifies the feasibility of mmlevel imaging resolution on portable devices for both 2D and 3D objects placed in a box.

35

Joint efforts from both academia and industry are needed to address further challenges
in the system level evaluation of ISAC, new channel modeling methodology, newwaveform design, low complexity algorithm design, and low-cost hardware design.
REFERENCE[1] Integrated Sensing and Communication (ISAC) – From Concept to Practice,https://www.huawei.com/en/huaweitech/future-technologies/integrated-sensingcommunication-concept-practice[2] W. Tong, P. Zhu, et al., "6G: the next horizon: from connected people and things toconnected intelligence," Cambridge: Cambridge University Press, 2021.
[3] O. Li et al., "Integrated sensing and communication in 6G: a prototype of highresolution THz sensing on portable device," in European Conference on Networks andCommunications (EuCNC), 8-11 June, 2021.

36

VI. Space-Time Synchronization
Tetsuya Ido, Nobuyasu Shiga, Motoaki Hara, Yuichiro Yano,
Satoshi Yasuda, Ryuichi Ichikawa
NICT

Abstract—For mobile communication technology to transform from a means of manto-man communication to an infrastructure for various vertical sectors in society, themethod of the time synchronization should shift from the traditional leader-follower
structure to autonomous distributed synchronization. Furthermore, synchronizationmust not only be limited to time but also extend to space, entailing the sharing(synchronization) of spatial coordinate axes. This would be realized by three basic
technologies, namely compact atomic clocks, wireless time synchronization, and cluster
clock systems. The combination will eventually acquire sensing capabilities like distancemeasurement through radio wave propagation time.
VI-1. Introduction
Networks have so far operated under the assumption that participant clocks are notsynchronized, establishing a kind of pseudo-synchronization on a one-to-one basis whennecessary, where signal transmission time was not taken into account. On the other
hand, GNSS has vividly demonstrated its high value by having participants (in this case,many GNSS satellites) with perfectly synchronized clocks, where signal transmission
time was taken into account. A prime example is the ability of positioning through radio
wave propagation time. Currently, mobile network base stations obtain time from GNSS,
but synchronization to GNSS time requires a certain duration, during which the local
clock frequency drifts, making it not easy to maintain a consistently synchronized clockin ns level. Additionally, synchronization through GNSS has limitations, such as being
unusable indoors and vulnerability due to weak satellite signal strength. Furthermore,
we should note that the heavy societal dependence on GNSS has led to considerations
for timekeeping methods independent of GNSS, as indicated by executive orders fromthe United States [1].
Beyond 5G networks are expected to transcend mere human communication andbecomeasocialinfrastructureforvariouseconomic

activities.

Space-timesynchronization proposed here will demonstrate solutions to the concern about theresiliency while enjoying the benefits of a synchronized network. This is based on:(1) Enhancing the basic strength of synchronized systems by implementing inexpensive,
miniaturized atomic clocks in base stations, autonomous vehicles, and even mobile
devices in future.

37

(2) Measuring time differences between clocks at nodes and achieving high-precisionsynchronization when necessary. Wireless time difference measurement or timesynchronization also enables distance measurement based on the signal propagationtime.
(3) Creating a virtual clock through the weighted average of local clocks at individual
nodes and sharing it across the network, thus generating a standard network clock withhigh resiliency.
VI-2. Three technologies comprising space-time synchronization
Toward 6G, ITU-R WP5D composed a document of technology aspects, “Futuretechnology trend for IMT-2030” [2]. This document clearly mentions that the threetechnologies are key for real-time communications/services. Here, we briefly describe
what they are, as follows.
VI-2.1. Chip-level integrated frequency standard (CLIFS)
Atomic clocks, with its ticking rate determined by atomic transition frequencies andknown for their extremely high frequency stability, became commercially available withthe size of a matchbox in 2010. However, mass production was not feasible, limiting theirwidespread use in consumer products. Recent advances in MEMS technology are about
making it possible to mass-produce atomic cells and GHz band oscillators with low phasenoise, leading to reduced costs. For instance, Fig. VI-1 shows mass-produced atomic cells
using silicon process technology. As a source oscillator, 3.4 GHz oscillator using a solidstate thin-film element (FBAR) recently achieves low phase noise below -124 dBc/Hz at
the power consumption of 3mW. The size projected currently ongoing is not yet suitablefor handsets but is sufficient for base stations, autonomous vehicles, and drones.

Fig. VI-1. Attempt for mass production of alkali-atom cells using MEMS technology.

38

VI-2.2. Wireless Two-way Interferometry (Wi-Wi)
To achieve high precision in comparing local clock times, signals are sentbidirectionally between nodes, enabling the measurement of both clock difference andsignal propagation time. Wireless Two-way Interferometry (Wi-Wi) achieves bothsyntonization (=identical frequency) and synchronization through the carrier wave of
wireless communication. Wi-Wi first compares the carrier phase difference between thelocal clocks of leader and follower modules. Phase locking (precise syntonization) is
accomplished as the follower module stabilizes its local clock to the leader clock. After
clock drift is corrected by phase stabilization, the clock is compared through packet
transmission and arrival timings. The follower module can then adjust its clock to theleader clock for synchronization.
We have developed a module incorporating a commercial off-the-shelf 920MHz RF chip,
fully aligned with the IEEE 802.15.4g standard. It can stabilize the phase with a jitterof ~20 ps and then synchronize the clock to 30 ns. Fig. VI-2 shows the latest version of
our Wi-Wi module, where the board size is 67.25mm x 31.75mm.
Two-way measurement provides the significant byproduct of ranging, stemming fromthe propagation time measurement. Phase measurement allows us to measure distancevariation at the sub-cm level. While there is room for improvement in synchronization
error, combining it with wide-bandwidth signals can feasibly suppress it to the sub-ns
level.

Fig. VI-2. Wi-Wi module.
VI-2.3. Cluster clock system
With local clocks having high frequency stability, it may not be the best way tosynchronize with an external master clock with a wide servo bandwidth. Rather, making
slight frequency adjustments to the atomic clock within a low bandwidth might yield amore stable and reliable clock. Also, creating a virtual clock by locally sharing a weighted
mean of the many clocks within the network enhances frequency stability and reduces

39the risk that specific clock failures affect whole network. Fig. VI-3 shows a POC setup
where ten nodes, each equipped with compact atomic clocks, are installed within thesame rack and interconnected by optical fiber, realizing a wired cluster clock architecture.
A virtual clock is generated through numerical processing from the time difference
between the neighbor clocks. Adding frequency and phase offset to the signal of local
free-running atomic clocks, each node can generate the ensemble clock as real signals.

Fig. VI-3. Demonstration of a cluster clock.
Left: Ten nodes equipped with compact atomic clock are connected in a star topology.
Right: Records of clock reading in one day. Cluster clock (red curve) shows an enhanced
stability, whereas ten thin curves are those of each clock.
VI-3. Space-Time Synchronization
Traditionally, synchronization often has involved a leader-follower model where theleader's signal is followed by the followers, including delay as well as noise within the
propagation delay, signifying that the time coordinate axis between the leader andfollower are not shared. This indicates that synchronization is about sharing the time
coordinate axis. Extending this concept to space constitutes the idea of space-timesynchronization.

Wi-Wi,

bymeasuringpropagation

delays,enablesdistancemeasurements. Furthermore, combining multiple units allows for positioning. Thisimplies that the spatial coordinate axes are shared among network participants, leading
to applications like multiple machines coordinating work based on the information ofmutual positioning, or multiple vehicles and pedestrians on roads moving coherently to
avoid collisions.

40

VI-4. Conclusion
The concepts of space-time synchronization and three key technology that realizes
space-time synchronization are briefly described. This idea will make the mobile
communication system more robust, resilient, and could make energy efficient.
Acknowledgements
This paper includes the results of research and development conducted by the Ministryof Internal Affairs and Communications (MIC) under its ”Research and Development for
Expansion of Radio Resources (JPJ000254)” program.
REFERENCE[1] Strengthening National Resilience Through Responsible Use of Positioning,
Navigation, and Timing Services, U. S. Executive order 13905 (2020)
https://www.federalregister.gov/documents/2020/02/18/2020-03337/strengtheningnational-resilience-through-responsible-use-of-positioning-navigation-and-timing[2] Future technology trends of terrestrial International Mobile Telecommunications
systems towards 2030 and beyond, ITU-R WP5D Rep ITU-R M. 2516-041

Abbreviation List
Abbreviation

Explanation

ABG

Alpha-Beta-Gamma

AI

Artificial Intelligence

ALD

Atomic Layer Deposited

AMC

Adaptive Modulation and Coding

AoA

Angle of Arrival

AR

Augmented Reality

ASIC

Application Specific Integrated Circuit

AWG

Arbitrary Waveform Generator

BAN

Body Area Network

BCB

Benzo cyclobutene

BER

Bit Error Rate

BF

BeamForming

BS

Base Station

CC

Component Carrier

CI

Close-in

CMOS

Complementary Metal Oxide Semiconductor

CPS

Cyber Physical System

CSI

Channel State Information

DC

Direct Current

DFT

Discrete Fourier Transform

DL

Down Link

DNN

Deep Neural Network

DOA

Direction of Arrival

DSP

Digital Signal Processing

EIRP
EVM

Error Vector Magnitude

eWLB

embedded Wafer Level Ball grid array

FDD

Frequency Division Duplex

FDE

Frequency Domain Equalize42Abbreviation

Explanation

FSPL

Free Space Path Loss

HARQ

Hybrid Automatic Repeat Request

HPBW

Half Power Beam Width

IBO

Input Back Off

IFFT

Inverse Fast Fourier Transform

InH

Indoor hotspot cell

ISAC

Integrated Sensing and Communication

ITU-R

International
Telecommunication
Radiocommunication Sector
Key Performance Indicator

KPI
LAN

Local Area Network

LNA

Low-Noise Amplifier

LOS

Light of Sight

LTE

Long Tern Evolution

MCM

Multichip Module

MIMO

Multiple-Input and Multiple-Output

MMIC

Monolithic Microwave IC

MS

Mobile Station

MOS

Metal Oxide Semiconductor

MOS-HEMTMSL

Metal-Oxide-Semiconductor
Transistor
Microstrip Line

Eigh-Electron-Mobility

NLOS

Non-Line of Sight

NR

New Radio

NRNT

New Radio Network Topology

OAM

Orbital Angular Momentum

OFDM

Orthogonal Frequency Division Multiplexing

PA

Power Amplifier

PAE
PCB

Printed Circuit Board

PLE

Path Loss Exponent

QMH

Qualitative Microwave Holography

RAN

Radio Access Network

43

Union

Abbreviation

Explanation

RAT

Radio Access Technology

RD

Relay Device

RFRadio Frequency

RIS

Reconfigurable Intelligent Surface

RMSE

Root Mean Square Error

RS

Relay Station

Rx

Receiver

SAG

Selective-Area Growth

SC

Single Carrier

SiP

System-in-Package

SISO

Single-Input Single-Output

SIW

Substrate-Integrated Waveguide

SNR

Signal to Noise power Ratio

TDD

Time Division Duplex

TDS

Time Domain Spectroscopy

THz

Tera Hertz

TMA

Trimethylaluminum

TSV

Through-silicon Via

Tx

Transmitter

UCA

Uniform Circular ArrayUE

User Equipment

UL

Up LinkVR

Virtual Reality

44