# RF-Motion: A Device-Free RF-Based Human Motion Recognition System

Authors: Jumin Zhao; Jianyi Zhou; Yuxuan Yao; Deng-ao Li; Liye Gao
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

In recent years, human motion recognition, as an important application of the intelligent perception of the Internet of Things, has received extensive attention. Many applications benefit from motion recognition, such as motion monitoring, elderly fall detection, and somatosensory games. Several existing RF-based motion recognition systems are susceptible to multipath effects in complex environments, resulting in lower recognition accuracy and difficulty in extending to other scenarios. To address this challenge, we propose RF-Motion, a device-free commercial off-the-shelf (COTS) RFID-based human motion recognition system that can detect human motion in complex multipath environments such as indoor environments. And when the environment changes, RF-Motion still has high recognition accuracy, even without retraining. In addition, we use data slicing to solve the problem of discontinuity in the time domain of RFID communication and then use the synthetic aperture (SAR) algorithm to obtain the fingerprint feature matrix corresponding to each motion. Finally, the dynamic time warping (DTW) algorithm is used to match the prior motion fingerprint database to complete the motion recognition. Experiments show that RF-Motion can achieve up to 90% accuracy for human motion recognition in an indoor environment, and when the environment changes, it can still reach a minimum accuracy of 87%.


---

## Body

## 1. Introduction

With the rapid development of the Internet of Things, human motion recognition has become a key technology in many emerging Internet of Thing applications. This is because the application of motion recognition technology in many fields can not only greatly enrich people’s lives but also facilitate people’s production activities. In recent years, motion recognition technology has been fully applied in the fields of smart home, human-computer interaction, and healthcare [1, 2].In recent years, motion recognition technology has received great attention from the academic community, and RFID technology has become a possible solution for motion recognition technology due to its advantages such as light weight, low cost, high sensitivity, and the ability to penetrate through walls. Some researchers have proposed tag-based motion recognition research [3, 4]. They require people to carry tags with them and use signal parameter changes brought by tag movement to achieve high-precision target positioning or behavior perception. The method does not meet people’s needs for comfort and convenience. Recently, a tag-free solution has been proposed [5, 6]; that is, people do not need to carry tags, but use the influence of the human body on the backscatter signal to identify motion.Today’s RFID tags provide very limited information. Existing RFID-based device-free motion recognition solutions mainly use raw RSS and phase values. However, prior experiments have shown that RSS readings are less sensitive to human movement and are not a valid indicator; although the phase angle is sensitive to motion, its accuracy is significantly affected by multipath, which unfortunately is inevitable in indoor environments [7]. The scheme using raw RSS and phase is far from satisfactory.In addition, the existing motion recognition scheme does not consider the influence of the multipath effect on the recognition effect. The influence of multipath on motion recognition is mainly reflected in two aspects: first, in a high multipath environment, due to the influence of multipath signals, the correlation between signal parameters such as RSS and phase and human motion is weakened, which affects the accuracy of recognition; in addition, multipath in different scenarios will be very different, which results in a system trained in one scene but not competent for recognition in other scenes. This greatly reduces the robustness of the motion recognition system.In this paper, we propose RF-Motion, an RFID-based device-free motion recognition system, which models RFID signals and then suppresses multipath to increase system robustness. According to the different signal interferences at each moment of motion, the characteristic information of the signal affected by motion is used as the fingerprint of motion recognition. Using RFID technology for motion recognition faces 3 key challenges:(1)
RF-Motion mainly works indoors. The indoor environment is mostly narrow, and the multipath effect is serious, which seriously affects the accuracy of motion recognition. And when the environment changes, the accuracy of motion recognition will be further reduced. Therefore, it is necessary to suppress multipath. In this paper, we propose a multipath suppression method based on background subtraction, which uses the characteristics that multipath remains relatively unchanged over time to model backscattered signals to eliminate multipath effects(2)
The communication protocol of the RFID system determines that the communication between each tag and the reader is discrete in the time domain. As shown in Figure1, only one tag can communicate at the same time, and the communication order of each round among multiple tags is randomly arranged. Based on the consideration of this feature, this paper combines the RFID positioning method and image recognition method to divide the collected data into pieces in time sequence during RFID motion recognition, which is equivalent to dividing a motion(3)
Because each motion has differences in starting time, speed, etc., the lengths of the two time series that need to be compared during matching may not be aligned on the time axis. Therefore, this paper uses the dynamic time warping (DTW) algorithm originally used for speech recognition to compare and normalize two time series and then judge the similarity between the two motionsFigure 1
Packet distribution in time domain.We summarize the contributions of this paper as follows:(1)
We design a device-free RFID motion recognition system that can be used in complex environments with high multipath and can maintain high recognition accuracy even when the scene changes. To our knowledge, we are the first to consider multipath in RFID motion recognition systems(2)
We propose an algorithm that can suppress multipath. Specifically, our algorithm can extract signals directly related to motion from signals that are flooded by multipath(3)
We propose a time slice-based motion segmentation method that can solve the problem of RFID unequal interval communication(4)
Finally, we implemented a system prototype RF-Motion for human motion recognition. We identified six predefined actions of 10 volunteers in three different experimental scenarios. The results show that the method can reach about 90% recognition accuracy in the initial environment. Even when the environment changes, the lowest recognition accuracy of 87% can still be obtained, which indicates that the method given in this paper is highly feasibleThe rest of the paper is organized as follows. Section2 introduces the research background in the field of motion recognition. Section 3 discusses the implementation details of the RF-Motion system. The experimental evaluation results of our method are described in Section 4. Finally, we review the related work in Section 5 and conclude the paper in Section 6.


## 2. RF-Motion System Design

### 2.1. RF-Motion System Overview

Since RFID RSS and phase are easily affected by multipath effects, the readings are inaccurate, which affect the accuracy of recognition. Therefore, background subtraction is used to suppress multipath and obtain clean readings that are not affected by multipath. Moreover, since the communication of the RFID is discrete in the time domain, the conventional recognition method based on the continuously changing signal feature cannot be used; so, the interference information of the motion-to-signal is preprocessed by the data frame method based on the data fragmentation. For each frame of data after preprocessing, the synthetic aperture radar (SAR) algorithm in the radar system is used to obtain the feature matrix corresponding to motion, which constitutes a prior knowledge base. Finally, the DTW algorithm is used to find the feature matrix generated by the new motion and the most matching feature matrix in the prior knowledge base, thus completing the recognition motion.The key technical components of the overall motion recognition method are shown in Figure2. The multipath suppression module uses the background subtraction method to eliminate multipath in order to preserve a clean channel. The priori fingerprint database is composed of a data preprocessing module and a feature information extraction module, and the data preprocessing module mainly processes the collected data into frames. That is, one motion is sampled at multiple sampling points, and multiple frames can be used to describe one motion. The feature information extraction module uses an angle of arrival (AoA) estimation for each frame of data according to the tag number, obtains a feature matrix for motion recognition, and determines corresponding feature points and physical coordinates for the data divided into frames. The SAR algorithm is used to estimate the characteristics of AoA acquisition signals. The feature information matching module uses the improved DTW algorithm to compare and normalize two time series, which minimizes the sum of costs, matches motion, and improves recognition accuracy.Figure 2
Overview of key technologies.


### 2.2. Multipath Suppression

The first step in the RF-Motion operation is to distinguish between human reflections and reflections from other objects in the environment, such as furniture and walls. Each reflector in the environment provides a component for the entire received signal. Generally speaking, the reflection of walls and furniture is much stronger than the reflection of humans. Unless these reflections are removed, they will mask the signal from humans so that the human motions cannot be perceived. In order to solve the above challenges, we model the backscattered signal by LOS propagation and use the background subtraction method to eliminate multipath according to the characteristics of the reflected signal.The signal received from the tag is usually represented as a complex stream. In theory, it can be expressed as(1)S=X⋅h,whereX is the binary bit stream modulated by the tag, and h=αeJθ is the channel parameter of the received signal. In the RFID system, we can obtain channel-related information that include RSS in dBm (denoted as A) and the phase value ϕ; so, the channel parameter h can be calculated as
(2)h=10A/10−3eJϕ.Figure3 shows the reflection model of the RFID system in a simple case. The RFID system consists of a reader (R) and a tag (T). The reader accesses the tag through radio frequency signals. The tag responds to the reader using backscatter technology. However, in actual application scenarios, the signals received by the tag are not only free space signals sent directly from the RFID antenna but also other multipath signals reflected by the environment. We abstract the main reflection sources into people (H) and other objects in the environment (E). Therefore, in the corresponding I-Q plane, the three received signals that are aliased at the tag end can be represented as Sfree, Sreflect, and Smutipath, respectively. Therefore, the actual signal received by the reader can be expressed as
(3)Sactual=Sfree+Sreflect+Smutipath.Figure 3
Reflection model of the tag.Here, the human motion causes the reflection path to change and thus affects theSreflect; so, the RSS and phase of the Sactual also change accordingly. In order to identify human motion, we need to separate the Sreflect from the received signal to roughly describe the distance of the reflection path. Specifically, we can estimate Sreflect by subtracting Sfree and Smutipath from Sactual, and we can measure Sfree and Smutipath in a static environment before humans enter the reading range for exercise.


### 2.3. Obtaining RFID Signal Feature

The RFID operating frequency is divided into low frequency (LF), medium frequency (MF), high frequency (HF), and ultra-high frequency (UHF). The RFID operating under UHF has the longest communication distance; so, UHF is used in the experiment. RFID. UHF RFID uses electromagnetic carrier communication of about 920 MHz. Its communication signal is an ordinary wireless communication signal. It has three basic properties: phase (ϕ), intensity (A), and frequency (f). The frequency is known, so as long as you know the phase and intensity of the signal that know the characteristics of the entire carrier signal.The data obtained by the RFID reader are phase (ϕ), intensity (A), tag number (ID), and time (T); then, the obtained information can be expressed as
(4)Antennaj=ϕ,A,ID,T,whereϕ=ϕr′,A=Ar′,ID=IDr′,T=Tr′, r, is the packet number,r=1,2,⋯,N; and j is the antenna number, j=1,2,⋯,N.


### 2.4. Extracting Motion Fingerprints Using RFID Phase Perturbation Features

Establish a prior knowledge base for matching the motion to be identified. Firstly, the phase and intensity data obtained by the motion of the RFID signal are fragmented, and then the feature vector corresponding to each frame of data is calculated to form the feature matrix of the motion. Similarly, the feature matrix of other motions can be obtained. Then, all the feature matrices of motion constitute a priori fingerprint library.


#### 2.4.1. Data Fragmentation

Since RFID communication is discrete in the time domain, there is no guarantee that each motion has a continuous signal. If it is directly matched, accidental errors may occur; so, the traditional recognition method based on continuously changing signal characteristics cannot be used. This paper is inspired by the concept of frame in the image recognition method, that is, the time sampling of a complete continuous motion. It is proposed that the data is divided into slices in time sequence when analyzing the data, that is, processed into frames, and the number of frames determines 1. The number of sampling points for motion. Framing, which is equivalent to dividing one motion we have collected into discrete moments, describes one motion by sequential position, and each position corresponds to one data frame.The data is collected in time series; so, the data inAntennaj is arranged in time order, and the order can be divided into an equal number of n parts; then, the data amount of each copy is k=N/n; thus, the data is divided into n frames, namely,
(5)Frameq=Antenna1q,Antenna2q,⋯,Antennajq.Among them, q=1,2,⋯,n.In this method, multiple tags can be used as the signal source and are separated after being framed; that is, eachFrameq data in Antenna is separated and classified by Tag ID, and the data corresponding to each tag is obtained:
(6)Tagd=Antenna1d,Antenna2d,⋯,Antennajd,whered is the tag ID number, and the structure of Frameq becomes
(7)Frameq=Tag1q,Tag2q,⋯,Tagdq.


#### 2.4.2. Calculate the Feature Matrix Corresponding to Motion

After dividing the data obtained by each antenna into multiple frames, the corresponding data information of each tag is separated. It is necessary to analyze the corresponding signal characteristics of each tag data ofFrameq. In this paper, the signal characteristics are obtained by using the AoA estimation method. The method steps are as follows:(1)Calculate the Phase Angle at Each AntennaThe data processed in this step is the data of a certain frame of each label. In the dataAntennaj, the data in the first column is the phase angle. Due to the environmental noise, the data may have certain fluctuations. In order to make the data statistically representative, the highest is adopted. The frequency data is real data, namely,
(8)ϕAntennaj=ϕ∣maxfrequenceϕr.(2)Calculate the Signal Expression Received by Each AntennaStep 1 has obtained the phase information of the data received by each antenna. According to the characteristics of the sine wave, the signal at timet can be expressed as
(9)SAntennaj=AAntennaj⋅expi×2πft+ϕAntennajCalculate the signal received by each antenna to form the signalS.(3)AoA EstimationThe matrixS is input into the SAR algorithm to obtain the AoA estimation. The parameters are antenna spacing Xd and angle value step Δλ (in degrees). The output data is
(10)B=Pm,m=Δλ,2Δλ,⋯,180Δλ,whereP is the estimated intensity of AoA.


### 2.5. Establishing a Priori Fingerprint Library

For each motion, there arel labels of data; so, each motion corresponds to l vectors B. The l vectors B are grouped into one group; that is, the feature matrix corresponding to one motion in the qth frame is formed:
(11)Action=B1,B2,⋯Bl.Perform the above operations on then-frame data to obtain the feature matrix of a certain motion:
(12)W=Action1,Action2,⋯Actionn.By inputting the data corresponding to all motions, the feature matrix corresponding to the captured motion is obtained, and then all the feature matrices of the motion constitute the knowledge baseDB for motion matching.


### 2.6. RFID Feature Comparison for Motion Recognition

When the motionx to be identified is made, the data of the M antennas are acquired. The data of the M antennas can be processed according to the methods of Sections 3.3 and 3.4 to obtain the Bx,Actionx and the characteristic matrix corresponding to the motion x.


#### 2.6.1. Identify Motion

That is, to find the feature matrix corresponding to a motion that best matchesWx in the DB, and the motion made by the user can be identified. For the matrix Action, each of the final column vectors corresponds to a smooth curve, and the matching finds a similar curve, in the actual application of motion recognition. Due to different personal habits, different users have different differences in arm length and motion duration when doing the same motion, which is the same as the problem of speech recognition. To solve this problem, this paper uses the DTW algorithm commonly used in speech recognition. The core idea of the DTW algorithm is to compare and reorganize the time series of the motion data to be recognized on the time axis and to map the input motion time axis to be recognized nonlinearly to the time axis of the prior knowledge base; so, that the alignment cost and minimum of all elements are minimized. Determine the similarity between the two and match.The DTW algorithm inputs two matrices corresponding to the curve to be matched, and the degree of similarity of the output curve can be obtained. For any element in a sequence pair,Actionα and Actionxβ, α∈1,μ, β∈1,μ, the Euclidean distance between the elements is the alignment cost, namely,
(13)Cα,β=Actionα−Actionxβ.Then, the sum of the costs of the sequence normalizationC is the matrix of μ×ν. Let Z be the alignment of the pairs of elements in matrix C, Z=z1,⋯,zh,⋯,zH, where maxμ,v≤H≤u+v−1 and zh=αh,βh. The DTW algorithm is to find the arrangement Z to minimize the cost C, namely,
(14)minZ∑h=1Hzh=∑h=1HCαh,βh.Z is the subject to the following constraints:
(1)
Boundary conditions:z1=0.0,zw=μ,ν(2)
Monotonic conditions:(15)αh+1≥αh,βh+1≥βh,αh+1+βh+1≥αh+βh,(3)
Window conditions:αh−βh≤Q,h=1,2,⋯.H.When the sequence-to-element difference is greater thanQ, Q=H/6 is generally taken. Then, the sequence pair is considered to correspond to 2 different motions. The window condition only requires the calculation of elements along the diagonal of the cost matrix C with a diagonal width of 2Q, and the computational complexity is greatly reduced.The sequenceWx obtained by a motion is matched with the feature matrix corresponding to each motion in the prior fingerprint database DB, and each column in the feature matrix of the motion to be recognized is smaller than the corresponding column of a motion feature matrix in the fingerprint library DB. The matching cost and the sum of the costs of all columns are the smallest. Then, the motion is considered to be the same motion as the corresponding motion in the knowledge base, that is, successful identification.


## 2.1. RF-Motion System Overview

Since RFID RSS and phase are easily affected by multipath effects, the readings are inaccurate, which affect the accuracy of recognition. Therefore, background subtraction is used to suppress multipath and obtain clean readings that are not affected by multipath. Moreover, since the communication of the RFID is discrete in the time domain, the conventional recognition method based on the continuously changing signal feature cannot be used; so, the interference information of the motion-to-signal is preprocessed by the data frame method based on the data fragmentation. For each frame of data after preprocessing, the synthetic aperture radar (SAR) algorithm in the radar system is used to obtain the feature matrix corresponding to motion, which constitutes a prior knowledge base. Finally, the DTW algorithm is used to find the feature matrix generated by the new motion and the most matching feature matrix in the prior knowledge base, thus completing the recognition motion.The key technical components of the overall motion recognition method are shown in Figure2. The multipath suppression module uses the background subtraction method to eliminate multipath in order to preserve a clean channel. The priori fingerprint database is composed of a data preprocessing module and a feature information extraction module, and the data preprocessing module mainly processes the collected data into frames. That is, one motion is sampled at multiple sampling points, and multiple frames can be used to describe one motion. The feature information extraction module uses an angle of arrival (AoA) estimation for each frame of data according to the tag number, obtains a feature matrix for motion recognition, and determines corresponding feature points and physical coordinates for the data divided into frames. The SAR algorithm is used to estimate the characteristics of AoA acquisition signals. The feature information matching module uses the improved DTW algorithm to compare and normalize two time series, which minimizes the sum of costs, matches motion, and improves recognition accuracy.Figure 2
Overview of key technologies.


## 2.2. Multipath Suppression

The first step in the RF-Motion operation is to distinguish between human reflections and reflections from other objects in the environment, such as furniture and walls. Each reflector in the environment provides a component for the entire received signal. Generally speaking, the reflection of walls and furniture is much stronger than the reflection of humans. Unless these reflections are removed, they will mask the signal from humans so that the human motions cannot be perceived. In order to solve the above challenges, we model the backscattered signal by LOS propagation and use the background subtraction method to eliminate multipath according to the characteristics of the reflected signal.The signal received from the tag is usually represented as a complex stream. In theory, it can be expressed as(1)S=X⋅h,whereX is the binary bit stream modulated by the tag, and h=αeJθ is the channel parameter of the received signal. In the RFID system, we can obtain channel-related information that include RSS in dBm (denoted as A) and the phase value ϕ; so, the channel parameter h can be calculated as
(2)h=10A/10−3eJϕ.Figure3 shows the reflection model of the RFID system in a simple case. The RFID system consists of a reader (R) and a tag (T). The reader accesses the tag through radio frequency signals. The tag responds to the reader using backscatter technology. However, in actual application scenarios, the signals received by the tag are not only free space signals sent directly from the RFID antenna but also other multipath signals reflected by the environment. We abstract the main reflection sources into people (H) and other objects in the environment (E). Therefore, in the corresponding I-Q plane, the three received signals that are aliased at the tag end can be represented as Sfree, Sreflect, and Smutipath, respectively. Therefore, the actual signal received by the reader can be expressed as
(3)Sactual=Sfree+Sreflect+Smutipath.Figure 3
Reflection model of the tag.Here, the human motion causes the reflection path to change and thus affects theSreflect; so, the RSS and phase of the Sactual also change accordingly. In order to identify human motion, we need to separate the Sreflect from the received signal to roughly describe the distance of the reflection path. Specifically, we can estimate Sreflect by subtracting Sfree and Smutipath from Sactual, and we can measure Sfree and Smutipath in a static environment before humans enter the reading range for exercise.


## 2.3. Obtaining RFID Signal Feature

The RFID operating frequency is divided into low frequency (LF), medium frequency (MF), high frequency (HF), and ultra-high frequency (UHF). The RFID operating under UHF has the longest communication distance; so, UHF is used in the experiment. RFID. UHF RFID uses electromagnetic carrier communication of about 920 MHz. Its communication signal is an ordinary wireless communication signal. It has three basic properties: phase (ϕ), intensity (A), and frequency (f). The frequency is known, so as long as you know the phase and intensity of the signal that know the characteristics of the entire carrier signal.The data obtained by the RFID reader are phase (ϕ), intensity (A), tag number (ID), and time (T); then, the obtained information can be expressed as
(4)Antennaj=ϕ,A,ID,T,whereϕ=ϕr′,A=Ar′,ID=IDr′,T=Tr′, r, is the packet number,r=1,2,⋯,N; and j is the antenna number, j=1,2,⋯,N.


## 2.4. Extracting Motion Fingerprints Using RFID Phase Perturbation Features

Establish a prior knowledge base for matching the motion to be identified. Firstly, the phase and intensity data obtained by the motion of the RFID signal are fragmented, and then the feature vector corresponding to each frame of data is calculated to form the feature matrix of the motion. Similarly, the feature matrix of other motions can be obtained. Then, all the feature matrices of motion constitute a priori fingerprint library.


### 2.4.1. Data Fragmentation

Since RFID communication is discrete in the time domain, there is no guarantee that each motion has a continuous signal. If it is directly matched, accidental errors may occur; so, the traditional recognition method based on continuously changing signal characteristics cannot be used. This paper is inspired by the concept of frame in the image recognition method, that is, the time sampling of a complete continuous motion. It is proposed that the data is divided into slices in time sequence when analyzing the data, that is, processed into frames, and the number of frames determines 1. The number of sampling points for motion. Framing, which is equivalent to dividing one motion we have collected into discrete moments, describes one motion by sequential position, and each position corresponds to one data frame.The data is collected in time series; so, the data inAntennaj is arranged in time order, and the order can be divided into an equal number of n parts; then, the data amount of each copy is k=N/n; thus, the data is divided into n frames, namely,
(5)Frameq=Antenna1q,Antenna2q,⋯,Antennajq.Among them, q=1,2,⋯,n.In this method, multiple tags can be used as the signal source and are separated after being framed; that is, eachFrameq data in Antenna is separated and classified by Tag ID, and the data corresponding to each tag is obtained:
(6)Tagd=Antenna1d,Antenna2d,⋯,Antennajd,whered is the tag ID number, and the structure of Frameq becomes
(7)Frameq=Tag1q,Tag2q,⋯,Tagdq.


### 2.4.2. Calculate the Feature Matrix Corresponding to Motion

After dividing the data obtained by each antenna into multiple frames, the corresponding data information of each tag is separated. It is necessary to analyze the corresponding signal characteristics of each tag data ofFrameq. In this paper, the signal characteristics are obtained by using the AoA estimation method. The method steps are as follows:(1)Calculate the Phase Angle at Each AntennaThe data processed in this step is the data of a certain frame of each label. In the dataAntennaj, the data in the first column is the phase angle. Due to the environmental noise, the data may have certain fluctuations. In order to make the data statistically representative, the highest is adopted. The frequency data is real data, namely,
(8)ϕAntennaj=ϕ∣maxfrequenceϕr.(2)Calculate the Signal Expression Received by Each AntennaStep 1 has obtained the phase information of the data received by each antenna. According to the characteristics of the sine wave, the signal at timet can be expressed as
(9)SAntennaj=AAntennaj⋅expi×2πft+ϕAntennajCalculate the signal received by each antenna to form the signalS.(3)AoA EstimationThe matrixS is input into the SAR algorithm to obtain the AoA estimation. The parameters are antenna spacing Xd and angle value step Δλ (in degrees). The output data is
(10)B=Pm,m=Δλ,2Δλ,⋯,180Δλ,whereP is the estimated intensity of AoA.


## 2.4.1. Data Fragmentation

Since RFID communication is discrete in the time domain, there is no guarantee that each motion has a continuous signal. If it is directly matched, accidental errors may occur; so, the traditional recognition method based on continuously changing signal characteristics cannot be used. This paper is inspired by the concept of frame in the image recognition method, that is, the time sampling of a complete continuous motion. It is proposed that the data is divided into slices in time sequence when analyzing the data, that is, processed into frames, and the number of frames determines 1. The number of sampling points for motion. Framing, which is equivalent to dividing one motion we have collected into discrete moments, describes one motion by sequential position, and each position corresponds to one data frame.The data is collected in time series; so, the data inAntennaj is arranged in time order, and the order can be divided into an equal number of n parts; then, the data amount of each copy is k=N/n; thus, the data is divided into n frames, namely,
(5)Frameq=Antenna1q,Antenna2q,⋯,Antennajq.Among them, q=1,2,⋯,n.In this method, multiple tags can be used as the signal source and are separated after being framed; that is, eachFrameq data in Antenna is separated and classified by Tag ID, and the data corresponding to each tag is obtained:
(6)Tagd=Antenna1d,Antenna2d,⋯,Antennajd,whered is the tag ID number, and the structure of Frameq becomes
(7)Frameq=Tag1q,Tag2q,⋯,Tagdq.


## 2.4.2. Calculate the Feature Matrix Corresponding to Motion

After dividing the data obtained by each antenna into multiple frames, the corresponding data information of each tag is separated. It is necessary to analyze the corresponding signal characteristics of each tag data ofFrameq. In this paper, the signal characteristics are obtained by using the AoA estimation method. The method steps are as follows:(1)Calculate the Phase Angle at Each AntennaThe data processed in this step is the data of a certain frame of each label. In the dataAntennaj, the data in the first column is the phase angle. Due to the environmental noise, the data may have certain fluctuations. In order to make the data statistically representative, the highest is adopted. The frequency data is real data, namely,
(8)ϕAntennaj=ϕ∣maxfrequenceϕr.(2)Calculate the Signal Expression Received by Each AntennaStep 1 has obtained the phase information of the data received by each antenna. According to the characteristics of the sine wave, the signal at timet can be expressed as
(9)SAntennaj=AAntennaj⋅expi×2πft+ϕAntennajCalculate the signal received by each antenna to form the signalS.(3)AoA EstimationThe matrixS is input into the SAR algorithm to obtain the AoA estimation. The parameters are antenna spacing Xd and angle value step Δλ (in degrees). The output data is
(10)B=Pm,m=Δλ,2Δλ,⋯,180Δλ,whereP is the estimated intensity of AoA.


## 2.5. Establishing a Priori Fingerprint Library

For each motion, there arel labels of data; so, each motion corresponds to l vectors B. The l vectors B are grouped into one group; that is, the feature matrix corresponding to one motion in the qth frame is formed:
(11)Action=B1,B2,⋯Bl.Perform the above operations on then-frame data to obtain the feature matrix of a certain motion:
(12)W=Action1,Action2,⋯Actionn.By inputting the data corresponding to all motions, the feature matrix corresponding to the captured motion is obtained, and then all the feature matrices of the motion constitute the knowledge baseDB for motion matching.


## 2.6. RFID Feature Comparison for Motion Recognition

When the motionx to be identified is made, the data of the M antennas are acquired. The data of the M antennas can be processed according to the methods of Sections 3.3 and 3.4 to obtain the Bx,Actionx and the characteristic matrix corresponding to the motion x.


### 2.6.1. Identify Motion

That is, to find the feature matrix corresponding to a motion that best matchesWx in the DB, and the motion made by the user can be identified. For the matrix Action, each of the final column vectors corresponds to a smooth curve, and the matching finds a similar curve, in the actual application of motion recognition. Due to different personal habits, different users have different differences in arm length and motion duration when doing the same motion, which is the same as the problem of speech recognition. To solve this problem, this paper uses the DTW algorithm commonly used in speech recognition. The core idea of the DTW algorithm is to compare and reorganize the time series of the motion data to be recognized on the time axis and to map the input motion time axis to be recognized nonlinearly to the time axis of the prior knowledge base; so, that the alignment cost and minimum of all elements are minimized. Determine the similarity between the two and match.The DTW algorithm inputs two matrices corresponding to the curve to be matched, and the degree of similarity of the output curve can be obtained. For any element in a sequence pair,Actionα and Actionxβ, α∈1,μ, β∈1,μ, the Euclidean distance between the elements is the alignment cost, namely,
(13)Cα,β=Actionα−Actionxβ.Then, the sum of the costs of the sequence normalizationC is the matrix of μ×ν. Let Z be the alignment of the pairs of elements in matrix C, Z=z1,⋯,zh,⋯,zH, where maxμ,v≤H≤u+v−1 and zh=αh,βh. The DTW algorithm is to find the arrangement Z to minimize the cost C, namely,
(14)minZ∑h=1Hzh=∑h=1HCαh,βh.Z is the subject to the following constraints:
(1)
Boundary conditions:z1=0.0,zw=μ,ν(2)
Monotonic conditions:(15)αh+1≥αh,βh+1≥βh,αh+1+βh+1≥αh+βh,(3)
Window conditions:αh−βh≤Q,h=1,2,⋯.H.When the sequence-to-element difference is greater thanQ, Q=H/6 is generally taken. Then, the sequence pair is considered to correspond to 2 different motions. The window condition only requires the calculation of elements along the diagonal of the cost matrix C with a diagonal width of 2Q, and the computational complexity is greatly reduced.The sequenceWx obtained by a motion is matched with the feature matrix corresponding to each motion in the prior fingerprint database DB, and each column in the feature matrix of the motion to be recognized is smaller than the corresponding column of a motion feature matrix in the fingerprint library DB. The matching cost and the sum of the costs of all columns are the smallest. Then, the motion is considered to be the same motion as the corresponding motion in the knowledge base, that is, successful identification.


## 2.6.1. Identify Motion

That is, to find the feature matrix corresponding to a motion that best matchesWx in the DB, and the motion made by the user can be identified. For the matrix Action, each of the final column vectors corresponds to a smooth curve, and the matching finds a similar curve, in the actual application of motion recognition. Due to different personal habits, different users have different differences in arm length and motion duration when doing the same motion, which is the same as the problem of speech recognition. To solve this problem, this paper uses the DTW algorithm commonly used in speech recognition. The core idea of the DTW algorithm is to compare and reorganize the time series of the motion data to be recognized on the time axis and to map the input motion time axis to be recognized nonlinearly to the time axis of the prior knowledge base; so, that the alignment cost and minimum of all elements are minimized. Determine the similarity between the two and match.The DTW algorithm inputs two matrices corresponding to the curve to be matched, and the degree of similarity of the output curve can be obtained. For any element in a sequence pair,Actionα and Actionxβ, α∈1,μ, β∈1,μ, the Euclidean distance between the elements is the alignment cost, namely,
(13)Cα,β=Actionα−Actionxβ.Then, the sum of the costs of the sequence normalizationC is the matrix of μ×ν. Let Z be the alignment of the pairs of elements in matrix C, Z=z1,⋯,zh,⋯,zH, where maxμ,v≤H≤u+v−1 and zh=αh,βh. The DTW algorithm is to find the arrangement Z to minimize the cost C, namely,
(14)minZ∑h=1Hzh=∑h=1HCαh,βh.Z is the subject to the following constraints:
(1)
Boundary conditions:z1=0.0,zw=μ,ν(2)
Monotonic conditions:(15)αh+1≥αh,βh+1≥βh,αh+1+βh+1≥αh+βh,(3)
Window conditions:αh−βh≤Q,h=1,2,⋯.H.When the sequence-to-element difference is greater thanQ, Q=H/6 is generally taken. Then, the sequence pair is considered to correspond to 2 different motions. The window condition only requires the calculation of elements along the diagonal of the cost matrix C with a diagonal width of 2Q, and the computational complexity is greatly reduced.The sequenceWx obtained by a motion is matched with the feature matrix corresponding to each motion in the prior fingerprint database DB, and each column in the feature matrix of the motion to be recognized is smaller than the corresponding column of a motion feature matrix in the fingerprint library DB. The matching cost and the sum of the costs of all columns are the smallest. Then, the motion is considered to be the same motion as the corresponding motion in the knowledge base, that is, successful identification.


## 3. Performance Evaluation

### 3.1. Experimental Setup

The RF-Motion system is primarily implemented using COTS UHF RFID devices, including an Impinj Speedway Revolution R-420 reader, four S9028PCL directional antennas, and six Alien 9640 tags, and the system operates at 920.625 MHz. The reader extracts backscattered signal characteristics (RSS and phase) and tag ID and continuously reports to the background PC via the Ethernet cable. To improve accuracy and reduce array position error, The antennas are placed on an antenna mounting plate with a component spacing of 4 cm. We use the center of the antenna center or the fixed-frame antenna jack as the reference point. We choose six labels to be placed in front of the antenna array and try to ensure that the label is facing the antenna. It is located at the half height of the antenna (that is, at the same height as the half-height of the antenna, level). We distribute the labels in a readable range of the reader as a signal source. The experimental setup is shown in Figure4.Figure 4
Experimental setup.RF-Motion software is implemented entirely in C #. It consists of two parts: a data acquisition module and a data analysis module. The data acquisition module is integrated with the Octane SDK and is an extension of the LLRP toolkit. It supports continuous tag query and a read rate of 340 reads per second. The data analysis module is responsible for processing the raw data and identifying each motion. The software runs on a Lenovo PC equipped with an Intel Core i7-7500U 2.70GHz CPU and 8GB RAM.


### 3.2. Create the Fingerprint Library

There are 2 kinds of schemes: (1) each motion is done multiple times, then loop matching, and the error data that cannot match is removed. The rest is a library. In the matching process, as long as the unknown motion matches a certain data in the library, it can be considered as a match. (2) Each motion is done multiple times, loop matching, and a data set with a high degree of matching with each data is found as a knowledge base, and the matching link only matches this set of data. Compared with scheme 1, the accuracy of scheme 2 is slightly lower, but the calculation of the matching link is much smaller. Therefore, we choose scheme 2 to build a fingerprint database. We limit the human motion range to the specified area in the center of the tag array, because if the subject’s movement position deviates too much from the antenna polarization direction, it will have a nontrivial effect on the signal profile and reduce the recognition accuracy.Without loss of generality, we invited 20 different men and women, different heights, and different fat and thin experimenters to do 6 kinds of motion in the prescribed activity area squat (SQ), lunge (LG), sit-ups (SU), simultaneous knee and abdominal raise (SA), push-up (PU), and stand-trunk-bending (SB)), in Figure5. It includes both SQ, PU, and SB, which are very different, and SU and SA, which are very similar in motion. Do 10 times for each motion, input the acquired data, and get the prior knowledge base according to the process described in Section 3 and Option 2.Figure 5
Sketches of the tested human motion.


### 3.3. Motion Recognition

We use classification accuracy to evaluate the performance of the motion RF-Motion system. The classification accuracy is defined asNcorrect/Ntotal, where Ncorrect and Ntotal, respectively, represent the number of successfully recognized motions and the total number of motions in the test data set.In our performance evaluation of motion recognition, we first estimate the classification accuracy of each motion separately. Figure6 gives six kinds of motion fusion matrix. The minimum accuracy of 6 motions is 83%, the highest accuracy is 95%, and the average accuracy of all motions is 90%. Experiments that RF-Motion does have the ability to recognize human motion.Figure 6
Confusion matrix of human motion.We continue to analyze the data in the fusion matrix, and we found that(1)
SU and SA have the lowest classification accuracy of 84% and 83%, respectively, and 16% probability to mistake SU as SA, and 10% probability to mistake SA as SU. In fact, the SU and SA motions are very similar, while the RF-Motion tag array is sparse, the recognition resolution is low, and the similar motion is not good, indicating the performance of our system in the classification of similar motion. There is room for improvement(2)
We found that the classification accuracy of the motion (SU, SA, and PU) using the lying posture is significantly lower than the motion (SQ, LG, SB) using the standing posture. The reason for this phenomenon is that the lying motion reflection surface is small, which causes us to lose some motion features in depth, and our lying motion is performed on the label array; so, the accuracy of the lying motion is generally lowWe plan to improve system performance from both software and hardware: the software improves the EPC protocol and explores ways to increase reader read rates, thereby increasing the number of antennas or tags while maintaining read rates. Obtain more dimensional information and increase system recognition resolution; in hardware, we plan to improve the receiver receiving antenna, improve the antenna receiving gain, and enhance the reader’s receiving sensitivity to obtain finer-grained features.


### 3.4. Impact of Surroundings

We evaluated the motion recognition performance in three typical indoor environments: the library, the lab office, and the empty hall, which correspond to the high, medium, and low multipath environments. In the hall, 10 volunteers performed 10 predefined actions 10 times each time to establish a prior fingerprint database. Let RF-Motion perform motion recognition for volunteers in three scenarios to verify system performance in different environments. We also performed experiments in the same environment using GRfid proposed in [8] and RFIPad in [5]. The experimental results are shown in Figure 7. RF-Motion is minimally affected by the surrounding environment, even in high-multipath environments. In the library, there is still up to 87% of recognition accuracy, which is only 3% lower than in the hall. The experimental results are not affected by environmental changes. In contrast, GRfid and RFIPad have recognition accuracy similar to RF-Motion in the hall, but their recognition accuracy is significantly reduced when the environment changes.Figure 7
Comparison with other algorithms in different environments.


### 3.5. Impact of Motion Position

We further studied how the antenna-to-human distance and human’s direction on the antenna affect the recognition accuracy. As shown in Figure8, during this experiment, we asked three volunteers to perform motions at different antenna-to-human distances and different angles, and each motion is performed 30 times by each volunteer. We average the recognition accuracy of these volunteers. The results show that when the antenna-to-human distances change (the angle is fixed), the recognition accuracy changes moderately, and this result clearly shows that our recognition scheme is insensitive to the human-to-antenna distance. When volunteers stand at different positions at a fixed distance to form different angles, the accuracy gradually decreases as the angle becomes larger. Specifically, when the angle is 0 degrees, RF-Motion has the highest recognition accuracy of nearly 90%, and when the angle keeps increasing, such as when the angle is 30 degrees, RF-Motion can still reach nearly 85% accuracy. But when the angle is 60 degrees, the accuracy will drop to nearly 70%, and the lowest data will reach 68%. According to the results of our experiments, we recommend that users move as close to the specified area as possible, that is, the position of the antenna facing directly.Figure 8
Recognition accuracy over different angles and different distances.


## 3.1. Experimental Setup

The RF-Motion system is primarily implemented using COTS UHF RFID devices, including an Impinj Speedway Revolution R-420 reader, four S9028PCL directional antennas, and six Alien 9640 tags, and the system operates at 920.625 MHz. The reader extracts backscattered signal characteristics (RSS and phase) and tag ID and continuously reports to the background PC via the Ethernet cable. To improve accuracy and reduce array position error, The antennas are placed on an antenna mounting plate with a component spacing of 4 cm. We use the center of the antenna center or the fixed-frame antenna jack as the reference point. We choose six labels to be placed in front of the antenna array and try to ensure that the label is facing the antenna. It is located at the half height of the antenna (that is, at the same height as the half-height of the antenna, level). We distribute the labels in a readable range of the reader as a signal source. The experimental setup is shown in Figure4.Figure 4
Experimental setup.RF-Motion software is implemented entirely in C #. It consists of two parts: a data acquisition module and a data analysis module. The data acquisition module is integrated with the Octane SDK and is an extension of the LLRP toolkit. It supports continuous tag query and a read rate of 340 reads per second. The data analysis module is responsible for processing the raw data and identifying each motion. The software runs on a Lenovo PC equipped with an Intel Core i7-7500U 2.70GHz CPU and 8GB RAM.


## 3.2. Create the Fingerprint Library

There are 2 kinds of schemes: (1) each motion is done multiple times, then loop matching, and the error data that cannot match is removed. The rest is a library. In the matching process, as long as the unknown motion matches a certain data in the library, it can be considered as a match. (2) Each motion is done multiple times, loop matching, and a data set with a high degree of matching with each data is found as a knowledge base, and the matching link only matches this set of data. Compared with scheme 1, the accuracy of scheme 2 is slightly lower, but the calculation of the matching link is much smaller. Therefore, we choose scheme 2 to build a fingerprint database. We limit the human motion range to the specified area in the center of the tag array, because if the subject’s movement position deviates too much from the antenna polarization direction, it will have a nontrivial effect on the signal profile and reduce the recognition accuracy.Without loss of generality, we invited 20 different men and women, different heights, and different fat and thin experimenters to do 6 kinds of motion in the prescribed activity area squat (SQ), lunge (LG), sit-ups (SU), simultaneous knee and abdominal raise (SA), push-up (PU), and stand-trunk-bending (SB)), in Figure5. It includes both SQ, PU, and SB, which are very different, and SU and SA, which are very similar in motion. Do 10 times for each motion, input the acquired data, and get the prior knowledge base according to the process described in Section 3 and Option 2.Figure 5
Sketches of the tested human motion.


## 3.3. Motion Recognition

We use classification accuracy to evaluate the performance of the motion RF-Motion system. The classification accuracy is defined asNcorrect/Ntotal, where Ncorrect and Ntotal, respectively, represent the number of successfully recognized motions and the total number of motions in the test data set.In our performance evaluation of motion recognition, we first estimate the classification accuracy of each motion separately. Figure6 gives six kinds of motion fusion matrix. The minimum accuracy of 6 motions is 83%, the highest accuracy is 95%, and the average accuracy of all motions is 90%. Experiments that RF-Motion does have the ability to recognize human motion.Figure 6
Confusion matrix of human motion.We continue to analyze the data in the fusion matrix, and we found that(1)
SU and SA have the lowest classification accuracy of 84% and 83%, respectively, and 16% probability to mistake SU as SA, and 10% probability to mistake SA as SU. In fact, the SU and SA motions are very similar, while the RF-Motion tag array is sparse, the recognition resolution is low, and the similar motion is not good, indicating the performance of our system in the classification of similar motion. There is room for improvement(2)
We found that the classification accuracy of the motion (SU, SA, and PU) using the lying posture is significantly lower than the motion (SQ, LG, SB) using the standing posture. The reason for this phenomenon is that the lying motion reflection surface is small, which causes us to lose some motion features in depth, and our lying motion is performed on the label array; so, the accuracy of the lying motion is generally lowWe plan to improve system performance from both software and hardware: the software improves the EPC protocol and explores ways to increase reader read rates, thereby increasing the number of antennas or tags while maintaining read rates. Obtain more dimensional information and increase system recognition resolution; in hardware, we plan to improve the receiver receiving antenna, improve the antenna receiving gain, and enhance the reader’s receiving sensitivity to obtain finer-grained features.


## 3.4. Impact of Surroundings

We evaluated the motion recognition performance in three typical indoor environments: the library, the lab office, and the empty hall, which correspond to the high, medium, and low multipath environments. In the hall, 10 volunteers performed 10 predefined actions 10 times each time to establish a prior fingerprint database. Let RF-Motion perform motion recognition for volunteers in three scenarios to verify system performance in different environments. We also performed experiments in the same environment using GRfid proposed in [8] and RFIPad in [5]. The experimental results are shown in Figure 7. RF-Motion is minimally affected by the surrounding environment, even in high-multipath environments. In the library, there is still up to 87% of recognition accuracy, which is only 3% lower than in the hall. The experimental results are not affected by environmental changes. In contrast, GRfid and RFIPad have recognition accuracy similar to RF-Motion in the hall, but their recognition accuracy is significantly reduced when the environment changes.Figure 7
Comparison with other algorithms in different environments.


## 3.5. Impact of Motion Position

We further studied how the antenna-to-human distance and human’s direction on the antenna affect the recognition accuracy. As shown in Figure8, during this experiment, we asked three volunteers to perform motions at different antenna-to-human distances and different angles, and each motion is performed 30 times by each volunteer. We average the recognition accuracy of these volunteers. The results show that when the antenna-to-human distances change (the angle is fixed), the recognition accuracy changes moderately, and this result clearly shows that our recognition scheme is insensitive to the human-to-antenna distance. When volunteers stand at different positions at a fixed distance to form different angles, the accuracy gradually decreases as the angle becomes larger. Specifically, when the angle is 0 degrees, RF-Motion has the highest recognition accuracy of nearly 90%, and when the angle keeps increasing, such as when the angle is 30 degrees, RF-Motion can still reach nearly 85% accuracy. But when the angle is 60 degrees, the accuracy will drop to nearly 70%, and the lowest data will reach 68%. According to the results of our experiments, we recommend that users move as close to the specified area as possible, that is, the position of the antenna facing directly.Figure 8
Recognition accuracy over different angles and different distances.


## 4. Related Work

The motion recognition system has become increasingly popular as a basic solution for human-computer interaction. Current systems not only allow users to use specialized devices but also use natural body movements and contextually relevant information. These systems typically utilize various sensors available on the device for motion recognition, such as computer vision (cameras, cameras, etc.) [9], acoustics [10, 11], inertial sensors [12], vibration sensors [13], and light sensor [14]. However, these techniques still encounter many limitations in implementation, such as customization for specific applications, sensitivity to light, high installation costs or high equipment costs, the need for handheld devices, or the need to install additional sensors.Today, some studies attempt to identify motion that utilizes dedicated signals, such as WiFi. WiFi signals can be used not only for motion recognition but also for positioning [15], body recognition [16], vibration detection [17], etc. These WiFi-based systems operate by analyzing characteristic changes in wireless signals, such as analyzing changes in channel state information (CSI) or received signal strength indication (RSSI) caused by human motion [18].Due to the small size and low price of RFID tags, it is easy to popularize and practical applications, making RFID-based motion recognition technology a hot topic for scholars at home and abroad. The existing RFID-based motion recognition technology can carry an RFID tag through a target, and the receiving end recognizes the motion by analyzing the amplitude or phase change of the RFID tag. As proposed in [3], FEMO is attached to the sports equipment, and the Free-Weight Exercise is identified and evaluated according to the change of the phase value [4]. Five labels are used to make RF-glove, thereby identifying fine-grained gestures and handwritten characters. Yang et al. propose to locate the human body based on the COTS RFID technique via a device-free approach [19], which shows the potential of device-free sensing in the RFID system. More recently, GRfid [8], another device-free further based on RFID, is proposed to recognize the human gesture use. RFIPad [5] uses tag phase contour changes to recognize fine-grained gestures. However, the phase is easily affected by complex multipath signals in the environment, and it is difficult to recognize small movements on a large scale. Further, [6] uses a tag array to recognize handwritten letters and gestures with higher precision. In addition to these motion recognition studies, RFID has also been used for activity identification [20], material identification [21], etc.In order to get a better experience for users, this paper considers the influence of multipath on motion recognition. This paper proposes a multipath suppression method based on background subtraction, and according to the different interference of motion on the signal, the real-time phase and intensity information of the signal will be obtained by the reader. The data is processed by time slice to obtain the feature vector, and then the machine learning method is used for motion recognition to improve the recognition accuracy. In this method, the user does not need to carry additional equipment, and the RFID tag is inexpensive and simple to deploy.


## 5. Conclusion

Motion recognition is an important part of IoT intelligent perception. It has great application prospects in the fields of smart home and somatosensory games, bringing convenience to people’s lives. This paper proposes a motion recognition system based on RFID technology, which is low cost and easy to implement. According to the actual situation, we propose a background subtraction method to suppress multipath and propose a data fragmentation method to preprocess the data. Then, we use the SAR algorithm to obtain the feature vector and finally use the DTW algorithm for high-resolution gesture recognition. Through actual experiments, it is verified that the method proposed in this paper can indeed realize motion recognition. For further research, we also considered ways to improve the system performance by improving the EPC protocol and improving the receiver antenna acceptance gain. If the method becomes mature, it will play a great role in intelligent monitoring.



---

*Source: 1497503-2021-03-09.xml*