RFMamba: Frequency-Aware State Space Model for RF-Based Human-Centric Perception
Authors: Rui Zhang
Ruixu Geng
Yadong Li
Ruiyuan Song
Hanqin Gong
Dongheng Zhang
Yang Hu
Yan Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Human-centric perception
State space model
Radio frequency
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Human-centric perception with radio frequency (RF) signals has recently entered a new era of end-to-end processing with Transformers. Considering the long-sequence nature of RF signals, the State Space Model (SSM) has emerged as a superior alternative due to its effective long-sequence modeling and linear complexity. However, integrating SSM into RF-based sensing presents unique challenges including the fundamentally different signal representation, distinct frequency responses in different scenarios, and incomplete capture caused by specular reflection. To address this, we carefully devise a dual-branch SSM block that is characterized by adaptively grasping the most informative frequency cues and the assistant spatial information to fully explore the human representations from radar echoes. Based on these two branchs, we further introduce an SSM-based network for handling various downstream human perception tasks, named RFMamba. Extensive experimental results demonstrate the superior performance of our proposed RFMamba across all three downstream tasks. To the best of our knowledge, RFMamba is the first attempt to introduce SSM into RF-based human-centric perception.

Published as a conference paper at ICLR 2025
RFMAMBA: FREQUENCY-AWARS STATE SPACE MODEL FOR RF-BASED HUMAN-CENTRIC PERCEPTION
Rui Zhang', Ruixu Geng', Yadong LI, Ruiyuan Song', Hanqin Gong', Dongheng Zhang', Yang Hu', Yan Chen
'University of Science and Technology of China, PUniversity of Washington (rui zh, gengrui xu, rysong, hangin_ ) @mail uste. edu. en, yadonglie ie uww edu, { (dongheng, eeyhu, eecyan) @uste. edu. cn
ABSTRACT
Human-centric perception with radio frequency (RF) signals has recently entered new era of end-to-end processing with Transformers. Considering the long- sequence nature of RF signals, the State Space Model (SSM) has emerged as a superior alternative due its effective long-sequence modeling and linear complex- ity. However, integrating SSM into RF-based sensing presents unique challenges including the fundamentally different signal representation, distinct frequency re- sponses in different scenarios, and incomplete capture caused by specular reflection. To address this, we carefully devise a branch SSM block that is characterized by adaptively grasping the most informative frequency cues and the assistant spatial information to fully explore the human representations from radar echoes. Based these two branches, we further introduce an SSM-based network for handling various downstream human perception tasks, named RFMamba. Extensive experi- mental results demonstrate the superior performance of our proposed RFMamba across all three downstream tasks. To the best of our knowledge, RFMamba is the first attempt introduce SSM into RF-based human-centri perception.
INTRODUCTION
Despite significant advancements in vision-based human perception (Cao et al. 2023), optical cameras face fundamental limitations when perceiving human motion non-line- con-line-of--igee (NLoS) or low-lighting scenarios (Geng et al. [021 2022). In contrast, radio frequency (RF) signals are
illumination-robus. and can penetrate through non-metallic materials such as concrete walls (Li et
2024a by Song et al., [022a; Ding et al. 2023 Wang et al., (202)). Existing studies have utilized various RF signals for human sensing, including Stepped-Frequenee Continuous Wave (SFCW) radar, Trequency-Moduulate Continuous Wave (FMCW) radar, and WiFi. Among these, SFCW and FMCW signals are well-suited for fine- mme-grained human sensing because their larger bandwidth. Leveraging the signals reflecting off human body, victims trapped within collapsed structures, making it vital for anti-terrorism and rescue operations.
track suspects hidden inside building or quickly locate
Prior works RF-based numan-centrin perception (HCP) predominantly follow two paradigms. The first approach follows a multi-stage pipeline, where RF heatmaps are first generated from radar signals and subsequently fed into neural networks to perform human perception. This multi-stage process, however, requires complex data preprocessing that results in reduced efficiency. The second approach employs a single-stage framework, exemplified by the RadarFormer model proposed by (Zheng et al., 2023b). They utilize a Transformer- based architecture (Vaswani et al., 2017)) to directly extract human representations from radar echoes, thereby eliminating the need for intermediate heatmap generation. This end-to-end process enables more efficient handling of various HCP tasks.
However, due the long-sequence nature of radar echoes Geng al. (2024). single-stage framework that adopts Transformer-based architecture suffers from quadratic computational overhead, resulting in prohibitively high computational complexity. Therefore, a natural question arises: can a more dependencies across large-
efficient yet effective solution be developed to capture the sequence RF? The recently popular State Space Model (SSM) with linear complexity could be a
"Corresponding author.

---

Published as a conference paper at ICLR 2025
Frinay Region
Netwo Attenion
Well Ocstusion
Free Space
(a) Variance Distribution Acroes Different Frequency Ranges (1) Transformer-basee Method (c) Our Mamba-based Method
Figure 1: (a) illustrates the variance response across different frequency in the human motion scenarios, where higher variance indicates more captured human information. (b) and (c) illustrate the effective receptive field (ERF) of the transformer-based model and our SSM-based model, respectively. The ERF demonstrates the model's response to diverse frequency ranges. Notably, only the proposed RFMamba can adaptively select the most informative frequency cues for different scenarios.
promising answer to this question. Nevertheless, due to the inherent physical characteristics of RF signals, three key challenges arise when adapting the SSM architecture for RF-based sensing:
Signal Representation: The representation of RF signals is fundamentally different from images or text. RF signals consist of amplitude and phase information, with amplitude denoting target reflectivity while phase enables precise distance and velocity estimation.
The signals in different frequency bands have varying abilities to penetrate obstacles, resulting in different frequency responses between the free-space scenario and the wall- occlusion scenario. As shown in Figure I the human motion information tends to concentrate in all frequency bands in free space, while concentrating in low-frequency bands under wall occlusion. Specular Reflection:
Frequency Response:
The human body acts as a reflector rather than a scatterer in the lower RF frequency range, causing single RF echo can only capture partial human limbs. Such specular reflections render the problem of RF-based human perception inherently ill-posed.
Expanding upon these signal characteristics, we first attempt to extend Mamba from the perspective of frequency analysis and propose frequency- Mamba for RF-based HCP, termed RFMamba. Specifically, we devise RF-State Space Model (RF-SSM) block with two core branches modeling in the frequency domain and spatio-temporal domain, respectively. Due to the three-dimension nature of RF, the frequency modeling branch correlates frequency cues in the amplitude and phase dimensions, and six-way (imuni-ddmensiona) scanning strategy is adopted to ensure each element in the echoes integrates information from all other locations in different dimensions. Additionally, a frequency adaptive feed-forware network is designed to adaptively identify the most informative frequency cues. Aside from the frequency branch, we also develop spatio-temporal modeling branch to better facilitate the learning process from the perspective of the spatial domain. Finally, the frequency information and the spatial assistant information are comprehensively interacted to enhance RF-based human perception. With an elaborate model design, our model can handle variable input sequence lengths, relying on multiple echoes to alleviate the problem of specular reflection when using single echo. RFMamba outperforms state-of- of the-art methods by locating human key points with an average error of 5.06 cm, identifying person ID and action category with a mean average precision of 0.9991 and score of 0.9994, respectively.
Our main contribution can be summarized as:
We pioneer the first state space model for RF-based human perception, demonstrating the potential of Mamba for efficient yet effective global modeling in long-sequence RF signals.
We introduce novel RF-SSM block which integrates both frequency domain and domain modeling to effectively capture critical characteristics of RF signals for human perception.
We propose six-way scanning strategy in the frequency modeling branch, which ensures com- prehensive interaction of amplitude and phase information across all dimensions and is capable of adaptively selecting the most informative frequency cues.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Radio Frequency Sensing. Traditional RF sensing methods primarily rely on advanced signal processing techniques to develop explicit models that link signal variations to human behaviors. However, as the complexity of sensing tasks increases, these models often fall short. Consequently, learning-based RF sensing, driven by machine learning techniques, has emerged as promising alternative (Zhao et al. 2018a| Xie et al., 2023] Gong et al. 2024). Recent ad advancements have explored various RF modalities: WiFi-based methods like GoPose (Ren et al. (022) and Person-in-
WiFi 3D Yan et al.
2024) achieve accurate 3D pose estimation using commodity devices. MmWave
approaches such as mmPose-NI.P (Sengupta Cao, 2023) and HuPR (L.ee et al. 2023) leverage point clouds for precise pose reconstruction. Expanding on these, milliFlow (Ding et al., 2025)
enhances motion sensing with scene flow estimation, mmmifff (Fan et al. 2025) uses diffusion models to improve pose estimation in noisy environments, and RadarOec (Ding et al. 2024) processes 4D radar data to deliver robust 3D occupancy predictions. The MM-Fi dataset YYngg et al. (023) further accelerates progress in RF sensing by offering
benchmark for various perception tasks. UWB systems like UWB--Pse (Song et al.] 2022b) and MD-Pose (Zhou et al., 2023) exploit wide bandwidth for through-wall applications. While these methods have progressed, they often rely on complex preprocessing (Wu et al., 2022) or struggle with long sequence efficiency (Zheng et al. 2023b). Recent RadarFormer (Zheng al. 2023b) introduced end-to-end processing radar echoes but only utilized amplitude information. Our work addresses these limitations by introducing a novel state space model, efficiently processing both amplitude and phase information in long RF sequences.
State Space Models. State Space Models (SSMs) have emerged as powerful alternative to traditional architectures like Transformers (Peng et al.) 2024 Hann et al.) 2024 Ye et al.) [025 Jin et al.. 2025). particularly for tasks involving long-range dependencies due to their linear complexity with input length (Gu et al. 2020 (02T). Initial work such as $4 (Gu et al. (0022) and S5 (Smith et al., (023) laid
the foundation for deep state-space models, demonstrating their ability to model sequences efficiently. This was followed by the introduction of Mamba (Gu & Dao 2024), which enhanced SSMs by incorporating a selective scan mechanism and efficient hardware design, making it highly competitive in various domains. Recent works have adapted Mamba for diverse tasks, including vision (Zhu et al. 2024), medical imaging (Ma et al. 2024 Liu et al. 2024a), and video understanding (Chen et al. 2024). However, these adaptations primarily focus on single-modality data or specific domain applications. RFMamba distinguishes itself as the first to optimize SSMs for RF-hased human-centric perception. Unlike existing Mamba-based models, RFMamba addresses the unique challenges of RF signals through novel components like the RF-SSM block and mini-dimensional scanning strategy. This approach enables efficient processing of both amplitude and phase information in long RF sequences, a crucial aspect not addressed by Mamba models in other domains.
Spato Marba
FM-FFM
Sage
Suge
ComphiCern
Layerterr
Comple Car
SLG
Compbacook
J0-35M
FFM
Lupetiier
Uanakacerr L-t
Logfflon
Sauiial Mappirg
Puete
Merka
Spara- Teenuced
A, exp(hA).
Frequncty
b = Abe-s Bxc
71 - Ch2 Dz,
TAPFR
Elerrtuue Marie ttator
Dlanedies Adilion Tennago Paroneer
Figure 2: The overview of our state space model for RF-based HCP. RFMamba consists of a 3D patch embedding module, two-stage encoder, and a task-specific head.
3

---

Published as a conference paper at ICLR 2025
3 METHOD
3.1 PRELIMINARY
3.1.1 FREQUENCY AND SPATIAL ANALYSIS RF SENSING
In RF-based human sensing, Stepped-Frequenge Continuous Wave (SFCW) signals are widely used due their high range resolution and penetration capabilities. The SFCW signal model can be expressed as:
N-1 s(t)
Σ an e2r
(I)
where N is the number of frequency steps, On is the amplitude, and fn = fo + nAf is the frequency at the n-th step, with fo being the start frequency and Aʃ the frequency step size.
The received RF signal can be formulated X = R j. CAXF, where A represents the number of antennas and F the number of frequency samples. The corresponding amplitude spectrogram. A and phase spectrogram P can be extracted by:
A(u, v) = (2 v) (2(, v), P(u, v) = arctan I(u, v) R(u, 2)
(2)
where u and v indicate the indices in the frequency domain. The inverse Fourier transform converts X to the spatial domain S(h, w))
A-1F-1 S(h, w) VAF ΣΣ
j2z
X(u, vle
This transformation allows analysis in three critical dimensions: angle (h), range (w)) and velocity (from range variations across frames). Our RFMamba leverages this multi-dimensional information, incorporating both frequency and spatial domain features through the proposed RF-SSM block and omni-dimensional scanning strategy, enabling comprehensive modeling of human motion information.
3.1.2 STATE SPACE MODELS
As novel basic operation, State Space Models (SSMs) capture long-term dependencies similar to self-attention but benefit from linear complexity, which shows efficient scalability with input length. By leveraging the content of linear ordinary differential equations to translate one-dimensional inputs into outputs via latent states, SSMs operate with high efficiency. For system with input xxt) and output (t), SSMs can be formulated as a linear ordinary differential equation:
(t(t) = At()) + Bz((), y(t) = Ch(t) Dz(t),
where h(() is the hidden state that accumulates historical information, and (1(() describes its temporal evolution. A E N governs state transitions, B E R maps inputs to state updates, C E R projects states to outputs, and D E R! provides direct input-output connections. For the through- wall radar system, the SFCW signals are processed using zero order hold (ZOH) discretization, enabling adaptive scanning of RF data for human motion reconstruction. This adaptability is particularly useful in RF sensing to reconstruct human motion from long-sequence discrete RF signals.
3.2 OVERALL ARCHITECTURE
As is illustrated in Figure RFMamba is a multi-stage network consisting of 3D patch embedding module, a two-stage encoder, and task-specific head. Specifically, to alleviate the problem of fragmented target information in a single frame due to specular reflection, we input 7 sequential radar echoes with the shape F. E (AxxFxF) into the patch embedding module, which can preserve the structural and sequential relationships of echoes. Then, two-stage encoder is employed to obtain the nigh-dimensional representation. Each stage consists down-sampling layer and an RF-SSM block. The down-sampling operation is implemented using a complex convolution layer
4

---

Published as a conference paper at ICLR 2025
Figure 3: The proposed omni- mineesiinnsional scanning strategy. In the frequency domain, (a), (b), and (c) represent scanning methods along the slow-time, fast-time, and antenna dimensions, respectively, In the spatial domain, (a) corresponds to scanning along the temporal dimension, while (b) and (c) represent scanning along different spatial dimensions.
with kernel size of 1 x 3 x and stride of 1 resolution & 4. The RF-SSM block includes the frequency modeling branch which captures 2.
resulting in 2x down-sampiin wown--amlling of the
long-term dependencies from amplitude and phase domain, and the spatiotemporal modeling branch which extracts distance, angle, and velocity features. The outputs from these branches are integrated into a unified high-ding dimenniinnii representation, which processed by lightweight task-specific head to generate predictions for downstream tasks.
3.3 3D PATCH EMBEDDING MODULE
To better utilize the temporal (slow-ting time dimension) information, we merge the 12 sequential radar echoes the layout in Figure (a), which can realize fixed scanning interval along the temporal dimension. After the merge operation, the shape of F. becomes F, C(A-Ti)((FTT)) We adopt 2D complex convolution to project the input F into N non-overlapping 2D patches Fμ E C(H XN), where 32 and W 39. Note that the 2D patches are further flattened into 1D sequence. which can preserve the 2D structure of the radar echoes. Furthermore, to preserve the information regarding the spatial and temporal positions, we further incorporate a learnable spatial position embedding, denoted Ps E xIX alongside additional temporal position embedding, represented as P, CTXIX1 Then, Pa and Pa are expanded into the same shape T x x F by broadcast operation. The patches Fp are initially rearranged into the shape at T A F, after adding the learnable spatial and temporal position embedding, the patches F, are rearranged into the shape H W N. This procedure is encapsulated as:
F F Pp +
(5)
3.4 RF-STATE SPACE MODEL BLOCK
The detailed structure of RF-SSM block shown Figure 2 which contains two parallel frequency and spatiotemporal branches to extract frequency and spatiotemporal information. The frequency branch captures long-term dependency from the amplitude and phase of RF signals, while the spatiotemporal branch captures the tpapio-temporal information (range, angle, and velocity) from the RF signal with an additional FFT operation. The frequency information and the spatiotemporal information interact with each other generate the high-level semantie representations, significantly improving the performance of RF-hased human-centric perception.
3.5
FREQUENCY MODELING BRANCH
3.5.1
OMNI-DIMENSIONAL SCANNING MECHANISM
Existing Mamba architectures(Zhu et al. 2024 highly depend on scanning directions, such as scanning natural language from left to right and scanning natural images from four directions. The radar signals are 3-dimension data, consisting of the fast-time dimension with distance information, the antenna dimension with azimuth and elevation information, and the slow-time dimension with

---

Published as a conference paper at ICLR 2025
velocity information. To scale the Mamba for radar signals, we propose scanning strategy that encapsulates comprehensive onni-dimensional information while maintaining linear computational complexity. Specifically,
combined 12 consecutive radar echoes shown in Figure B The scanning strategy along the slow-time dimension is illustrated Figure 3 (a), and the scanning strategy along the fast-time dimension and antenna dimension is shown in Figure (b). The omni- dimensional scanning mechanism includes six scanning directions: forward antenna dimension, forward fast-time dimension, forward slow-time dimension, backward antenna dimension, backward fast-time dimension, and backward slow-time dimension. Subsequently, the features from six scanning directions are merged to form the final features, allowing the model effectively capture global frequency information.
3.5.2 FREQUENCY MAMBA MODULE
The feature map Fp in the frequency domaín consists of two parts, i.e., amplitude spectrogram A(F and the phase spectrogram PF)p). These two spectrograms are processed individually using the progressive frequency scanning branch depicted in Figure 2 obtain AA' A'(Fp) and P'(P,).
(6) where FregScan indicates the frequency scanning process. The detailed of FreqScan is shown in Figure 3 The FreqScan contains series of operations: Linear - DWCome + SiLU - SS2D + Layer.Norm > Linear. Unlike the vanilla scanning strategy (Liu et al. 2024b), our scanning strategy is the omni-dimensional frequency learning. The amplitude and phase scanning information are further integrated by frequency summarization module as
A' FreqScan(A(Fp)))
P'(Ep) FreqScan (PaaScaan((FF))))
Fp = Linear (A' (F₂)) + Linear (Fz))).
3.5.3 FA-FFN MODULE
Not all low- and high-frreueee information contributes latent human perception. To address this, we propose a Frequency Adaptive Feed-Forward Network (FA-FFN) module, which adaptively identifies the relevant frequency components focus on. The FA-FEN be formulated follows:
X3 DWConv
(F,) O Wftt), F = GELU(Xa) Xa-
(8)
The input feature map F, is first passed through a 1 x complex convolution layer, which projects into a higher-dimennsonal space.
The transformed features are element-wiss multiplied by a learnable weight matrix Wfft, followed by complex depthwise convolution (DWConv) to generate X3. Subsequently, the GELU activation function is applied to introduce non-linearity, and the result is further multiplied by Xs. producing the final output F,.
3.6 SPATIOTEMPORAL MODELING BRANCH
The heatmaps in the spatial domain are considered to contain latent information related to distance, angle, and velocity. In light of this, we design spatiotemporal modeling branch, which assists the learning process from a spatial domain perspective. The detailed structure of the spatiotemporal branch is shown in Figure
3.6.1 SPATIAL MAPPING MODULE
By converting radar echoes from the frequency domain into heatmaps in the spatial domain, the spatial mapping module, which utilizes both IFFT and FFT, enables detailed local analysis. The radar echoes captured by the radar system are denoted as F. CTxPxAxXA where A x A₂ = A. First, a ID IFFT is applied along the fast-time dimension to extract the target's 's distance information. Then, ID FFT is applied along the antenna dimensions A₁ and Ag to capture angle information of azimuth and elevation. The spatial mapping module performs these transformations along the fast-time dimension, transmitting antenna array, and receiving antenna array, respectively. The absolute value function is then applied to obtain the spatial heatmap F, formulated as:
(9) By stacking T' spatial heatmaps F., the spatiotemporal heatmap F Fst, which contains both spatial and temporal information, is obtained. The arrangement of multi spatial heatmaps can be seen in Figure 3 The F., is then fed into the spatiotemporal scanning module.
F ABS((FF (F.), FAT (Fa), (F.))).

---

Published as a conference paper at ICLR 2025
3.6.2 SPATIOTEMPORAL MAMBA MODULE
To achieve a better fusion of frequency components and spatial components, the layout of 12 consecutive spatial heatmaps and the scanning strategy are the same as those in the frequency scanning branch. Then the spatiotemporal heatmap Fst, which is a combination of multi spatial heatmaps F. is obtained. Following the design in (Zhu et al.) (002), the Spatiotemporal Mamba Module STSean is adopted to capture spatiotemporal features, formulated as
STScca (Fonv(F.)) @ SiL.U(F.):
(10)
where Conv represents a x convolution and STScan refers to the spatiotemporal Mamba mentioned earlier, the spatial Mamba module follows similar operational sequence to the frequency Mamba module. Since the Fourier domain possesses global properties, where each pixel in the Fourier space interacts with all spatial pixels, it is natural to explore the HCP task using Fourier transforms. Therefore, the FFT is employed to transform the spatial heatmaps from the spatial domain to the Fourier domain F,, which encapsulates global characteristics.
Finally, the outputs from the frequency and spatiotemporal modeling branches are concatenated, followed by 1 convolution harmonize the features. An element-wise addition of residual connection branch is then performed, yielding the final output.
3.7 Loss FUNCTION
To enable joint learning across multiple tasks, we define the total loss function Lootal, which consists of three components: pose estimation loss lopoee. action recognition loss Lactsons and RelD loss Lreid- For Lpose: we use the Mean Per Joint Position Error (MPJPE) (Ionescu et al. 2013). which calculates the average Euclidean distance between the predicted joints and the ground truth joints k. Both Liction and Lread are optimized using cross-entropy loss (Zhang Sabuneu| 2018). During evaluation, action recognition performance is assessed using accuracy and FI score, while RelD performance is measured using mean average precision (mAP) and the cumulative matching curve (CMC) rank-1. 1. Therefore, the joint training loss function can be formulated
Tsotal Lopose Lraction + Toycid-
(11)
Besides, we assess the model size by evaluating the total number of trainable parameters (Params).
THP DATASET
While numerous RF-based human perception datasets exist, they predominantly focus on free- space scenarios. The unique challenges posed by through-wall environments, such as signal at- tenuation and multipath propagation, necessitate a specialized dataset. To address this critical gap, introduce the Through-Wall Human- Centric Perception (THP) dataset, a comprehen- sive dataset for the through-wall human-centric perception tasks.
19
19
Data Collection System. We designed multi- modal data collection system, as illustrated in Figure comprising two primary components. The through-wall radar system, designed for transmitting and receiving RF signals to per- ceive human motion, consists of multi-input
Figure 4: Experiment scenario and multi-modal dataset collection system. Rx the receiving antennas, and Tx are the transmitting antennas.
multi-output (MIMO) antenna array and a vector network analyzer (VNA). The MIMO antenna array layout is optimized to mitigate sidelobe effects, effectively forming an array of 144 virtual antennas. The VNA generates range, enabling range resolution of approximately 75 mm.
continuous wave (SFCW) signals within the 0.8-2.8 GHz
To provide accurate ground truth labels, we implemented multi calti-cammra system with 12 nodes, each composed of Raspberry Pi, camera module, and Ethernet-powered module. We employ the
7

---

Published as a conference paper at ICLR 2025
calibration technique from (hang 2000) perform pairwise calibration between adjacent cameras, aligning all cameras to a unified world coordinate system.
THP Dataset Description. We introduce the THP dataset contains approximately 351,000 pairs of RF frames with corresponding 3D human keypoints, person identity labels, action category labels, and optical images. THP comprising two distinct subsets: THP-W (Wall-Occlusion) and THP-F (Free-space). (1) THP-W W consists of 255000 pairs of RF frames collected in wall-occlusion environments, featuring 10 individuals performing 19 action categories, including both dynamic actions and static poses. The primary obstacle is a 23 cm concrete wall, presenting a significant challenge for through-wall human perception methods. (2) THP-F consists of 96000 pairs of RF frames collected in free-space environments, maintaining similar data structure and action categories as THP-W. This subset allows for evaluating model performance without wall occlusion.
Both subsets share common characteristics, including scanning rate of 12 frames per second (FPS), single radar echo dimensions of 201 x 144, and optical image resolution of 1640 1248. The subjects in the dataset exhibit diverse physical characteristics, enhancing the robustness of the data. This comprehensive dataset enables rigorous evaluation of RF-based human sensing methods across various environmental conditions, from occluded to free-space scenarios, providing valuable resource for advancing research in this field.
EXPERIMENTAL RESULTS
Implementation Details. All baselines and our RFMamba are trained using an Nvidia RTX4090 GPU and implemented with PyTorch. We used the Adam optimizer with an initial learning rate of 2c-3, which decays by factor of 0.5 (gamma) every 10 epochs using the StepLR scheduler. The batch size was set to 50, and training epochs were set to 50 for all models except RadarFormer (1000 epochs due to slower convergence). For RFMamba, stacked 12 consecutive frames as input. The dataset was split into a 4:1 training-testing ratio, using a fixed random seed of 42.
Baselines. To demonstrate the superiority of the proposed RFMamba framework, we compare it with various RF-based human perception baselines, including (1) RadarFormer (Zheng et al.] 2023b), (2)
RFPose3D (Zhao al. 2018b), (3) mmPose (Sengupta al. (020), and (4) ResNet (Haral
et al. 2018). RadarFormer is the first end-to-end transformer network in through-wall scenarios. RFPose3D is the state-of state-of-the-ar(SSe (SOTTA) method for human pose estimation utilizing 2D horizontal
and vertical RF heatmaps. As majority of through-wall perception methods (Zheng et al. (223a))
can be considered as variants of ResNet3D, we choose ResNet3D-50 the backbone and MLP as the task-specifi heads to perform human perception using 3D radar heatmaps.
Method Nose
Neck Shoulder Elbow
Wrist Hip
Knee
Ankle
Eye
Ear
Mean (mm)
Params (M)
RI-Pose3D
81,30 62.98
78.95
96.93 122.06 75.84 80.47 83.44 78.63 79.67
85.35
10.91
ResNet3DD-0
105.26 86,10
98.38
114.51 161.95 88.62 87.35 97.82 106.13 97.60
105.34
352.30
mm-Pose 160.92
147.69
159.82
189.97
247.54 145.57
198.19
140.37
161.65 156.41
165.98
22.07
RadarFormer 281.26 239.97
248.36
277.24
344.03 231,88
23
208.42
280.02 258.29
258.90
12.88
RFMamba 51.85 41.31
44.03
52.73 68.89 41.53 46.32 55.18 53.30 47.24
50.64
1.94
Table 1: Quantitative Evaluation Results for Pose Estimation Task. The notation 'L': lower is better.
5.1 QUANTITATIVE RESULTS ON DIVERSE DOWNSTREAM TASKS
Performance of Pose Estimation. As shown in Table I RFMamba achieves state-of-the-art ahr MPJPE of 50.64 mm, significantly outperforming all baselines. This superior performance stems from three key innovations: (1) Comprehensive temporal modeling via 3D spatial-temporal volumes, surpassing RadarFormer's limited two-frame approach; (2) Advanced multi-scale feature extraction through synergistic frequency selective and spatiotemporal assistant modeling branch, addressing the temporal limitations of ResNet3D-500 and (3) Efficient information utilization via omni-dimensiooal frequency scanning strategy, preserving critical motion cues lost in RFPose3D's 2D heatmaps and mm-Pose's sparse point clouds. These architectural advancements enable RFMamba to effectively tackle the unique challenges of through-wall human pose estimation across diverse scenarios.
8

---

Published as a conference paper at ICLR 2025
Performance of Activity Recognition. Table 2 presents the quantitative comparison for human activity recognition. Existing methods show lim- itations in temporal modeling: ResNet31D-50 re- lies single-frame inputs, while RadarFormer uses only two successive frames, both resulting in suboptimal performance due insufficient temporal information. In contrast, RFMamba significantly outperforms these baselines by em- ploying onni-dimensional scanning mecha- nism that captures informative cues. Specifically, our model processes sequences of frames, striking an optimal balance between inference efficiency and performance. This strategy allows RFMamba to accurately identify complex action patterns, even in challenging through-wall scenarios.
Action Recognition
Person Re-ID
Method
Accuracy
FI Score
mAP
CMC-1
ResNetaD-50
0.9871
0.9820
0.8575
0.8979
RadarFormer
0.8918
0.7020
0.4817
0.8986
RFMamba
0.9997
0.9994
0,9991
0.9967
Table 2: Evaluation of Action Recognition and Person Re-ID.
Performance of Person ReID. Table 2 reveals strong correlation between ReID performance and accuracy in pose estimation and action recognition tasks. This correlation stems from the fact that individuals possess unique skeletal structures and exhibit distinctive behavioral patterns, which serve as crucial cues for RelD. Notably, RFMamba outperforms both CNN-based ResNet3D-S5 and Transformer-based RadarFormer in ReID tasks. This superior performance be attributed to RFMamba's s advanced temporal modeling and feature extraction capabilities, which effectively capture these individual-specinn characteristics.
(a) Wall-Occlusion Scenarios.
Free-Space Scenarios.
Figure 5: Qualitative results of 3D human pose estimation. Top row: RGB images showing various human actions. Bottom row: Corresponding 3D pose reconstructions by RFMamba.
5.2 QUALITATIVE ANALYSIS
Qualitative Analysis of 3D Joint Reconstruction. Figure Sa presents a qualitative evaluation of RFMamba in various wall-occlusive scenarios. RFMamba demonstrates superior accuracy in reconstructing detailed joint positions, particularly in challenging areas such as wrists and ankles.
Ca) CNN based ResNeCID-N
(b) Transformce based Radart oemer
Mambo based B.INamba
Figure 6: t-SNE visualization of learned features from different models. Each point represents feature vector, color-coded by action category. Note the superior cluster separation and definition in RFMamba, particularly for minority classes and similar action types.
Feature Space Visualization. We employ t-distributed Stochastic Neighbor Embedding (t-SNE) to project decoder output features onto a 2D plane, as shown Figure 6 RFMamba's 's encoder demonstrates superior feature separation, forming well-defined clusters even for minority classes and tail-end categories of our long-tailes distribution dataset. This capability particularly evident in static pose category 16, highlighting RFMamba' ability to learn discriminative features across diverse action types. In contrast, ResNet3D-S5 struggles to differentiate between dynamic scenarios like waving and walking, while RadarFormer fails distinguish similar minority poses (e.g., static poses

---

Published as a conference paper at ICLR 2025
5 and 14) from dynamic waving actions. These visualizations underscore RFMamba's effectiveness in capturing subtle, action-specific features, directly contributing to its superior performance in action recognition tasks.
5.3 GENERALIZATION CAPABILITY
To demonstrate the versatility of our approach, we evaluated its performance in free-space (unoc- cluded) settings using the -HP-F dataset. The model achieves MPJPE of 60.69 mm, mAP of 1.0 and an Fl score of 1.0. The results demonstrate effective generalization to conditions different from its primary training environment. Figure 5b provides qualitative evidence of accurate pose reconstructions and action classificatiinns in free-space environments. These results highlight the potential for applications where of-sihh is unimpeded.
Additionally, experiments indicate that the proposed method maintains high accuracy in multi person scenarios. Detailed results for multi-person environments are presented in Appendix A..I
The consistent performance across varying conditions from through- wall to free-space, and from single-person multi-person scenarios underscores the versatility of our approach and its potential for diverse real-world applications.
Pose
Action
Re-ID
Method
MPJPE (mm)
Accuracyt
mAPt
RFMamba
50.64
0.9994
0.9991
w/o FMB
9.007%(55.24)
-0.16% (0.9978)
-0.63% (0.9928)
w/o SMB
2.53%((51.92
-0.13% (0.9981)
-0.61% (0.9930)
w/o FA-FFA
1.14%%(51.22)
-0.51% (0.9943) +0.06% (0.9997)
Table 3: Ablation studies and analysis. The w/o indicates "without". FMB the frequency modeling branch. SMB is the spatiotemporal modeling branch. FA-FFA is the FA-FFN module.
5.4 ABLATION STUDY.
We conduct ablation studies to evaluate the effectiveness of three key components in RFMamba. Table[3 the results across all three downstream tasks.
Frequency Modeling Branch. This core component of the RF-SSM block correlates RF frequency information from amplitude and phase dimensions. Removing led significant performance drops across all tasks, with pose estimation eтror increasing by 9.07% %. This underscores the critical importance of frequency information modeling in RF-based tasks.
Spatiotemporal Modeling Branch. This branch enhances the learning process by modeling spa- tiotemporal features from the frequency analysis perspective. Its removal resulted in 2.53% increase in pose estimation error, 0.13% decrease in action recognition accuracy, and a 0.61 % decrease in person ReID accuracy.
The FA-FFN module adaptively identifies the most informative frequency cues, addressing the varying relevance of low and high-frequency information in human-centric perception tasks. Replacing it with vanilla FFN led to performance degradation in pose estimation and action recognition, while maintaining similar performance in person ReID. This validates the module's effectiveness in enhancing task-specific feature extraction.
FA-FFN Module.
CONCLUSION
This paper introduces RFMamba, novel approach for through-wall human sensing using RF signals. Our method incorporates an advanced architecture with omni-dimensional frequency scanning, sig- nificantly improving performance in pose estimation, action recognition, and person re-identification tasks. Comprehensive evaluations demonstrate RFMamba's 's superior performance and generalization capabilities across various scenarios, including free-space and multi-person person environments. These advancements open new possibilities for srrvacy-preservingg applications in healthcare, smart homes, and security systems.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by National Natural Science Foundation of China under Grants 62172381 and 62201542.
REFERENCES
Haozhi Cao, Yuecong Xu. Jianfei Yang, Pengyu Yin, Shenghai Yuan, and Lihua Xie. Multi-modal continual test-time adaptation for 3d semantic segmentation. In Proceedings of the IEEE/CVF 2023.
International Conference Computer Vision, pp.
Guo Chen, Yifei Huang, Jilan Xu, Baoqi Pei, Zhe Chen, Zhiqi Li, Jiahao Wang, Kunchang Li, Tong Lu, and Limin Wang. Video mamba suite: State space model as versatile alternative for video understanding. arXiv preprint arXiv:2403.09623 2024.
Fangqiang Ding, Andras Palffy, Dariu M Gavrila, and Chris Xiaoxuan Lu. Hidden gems: 4d radar scene flow learning using pross-modal supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9340-9349, 2023.
Fangqiang Ding, Xiangyu Wen, Yunzhou Zhu, Yiming Li, and Chris Xiaoxuan Lu. Robust 3d occupancy prediction with 4d imaging radar. Advances in Neural Information Processing Systems (NeurlPS), 2024.
Fangqiang Ding, Zhen Luo, Peijun Zhao, and Chris Xiaoxuan Lu. milliflow: Scene flow estimation mmwave radar point cloud for human motion sensing. In European Conference on Computer Vision, pp. 202-221.. Springer, 2025.
Junqiao Fan, Jianfei Yang. Yuecong Xu, and Lihua Xie. Diffusion model is good pose estimator from 3d vision. In European Conference on Computer Vision, pp. 1-18. Springer, 2025.
Ruixu Geng, Yang Hu, and Yan Chen. Recent advances on -on-line-of-sightt imaging: Conventional physical models, deep learning, and new scenes. APSIPA Transactions on Signal and Information Processing, 11(1), 2021. doi: 10.1561 15.1/111/111 16.0000100000011,
Ruixu Geng, Yang Hu, Zhi Lu, Cong Yu, Houqiang Li, Hengyu Zhang, and Yan Chen. Passive Non-Line-of-Sighe Imaging Using Optimal Transport. IEEE Transactions on Image Processing, 31:110-124, 2022. ISSN 1941-0042. doi: 10. 10.1109/1P22021.1228311
Ruixu Geng, Yadong Li, Dongheng Zhang, Jincheng Wu, Yating Gao, Yang Hu, and Yan Chen. Dream-ped: Deep reconstruction and enhancement of mmwave radar pointcloud. IEEE Transac- fions on Image Processing, 33:6774-6789, 2024. doi: 10.1 109/T11.2224..3123366
Hanqin Gong, Dongheng Zhang, Jinbo Chen, Yadong Li, Guixin Xu, Yuqin Yuan, Yang Hu, and Yan Chen. Enabling orientation free mmwave-basee vital sign sensing with multi-domain signal analysis. In ICASSP 2024-2024 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 8751-8755. IEEE, 2024.
Albert Gu and Tri Dao. Mamba: Linear- Time Sequence Modeling with Selective State Spaces, May 2024.
Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. HiPPO: Recurrent Memory with Optimal Polynomial Projections. In Advances Neural Information Processing Systems, volume 33, pp. 1474-14400 1487. Curran Associates, Inc., 2020.
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers. In Advances in Neural Information Processing Systems, volume 34, pp. 572-585. Curran Associates, Inc., 2021.
Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the Parameterization and Initialization of Diagonal State Space Models, Advances in Neural Information Processing Systems, 35:35971- 35983, December 2022.
11

---

Published as a conference paper at ICLR 2025
Xu Han, Linghao Jin, Xiaofeng Liu, and Paul Pu Liang. Progressive compstiiiliyy in text-to-image generative models, 2024. /ttpa///...////////////////////.
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE conference on Computer Vision and Pattern 2018.
Recognition, pp.
Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions pattern analysis and machine intelligence, 36(7): 1111133533333 2013.
Yi Jin, Xiaoxiao Ma, Rui Zhang, Huaian Chen, Yuxuan Gu, Pengyang Ling, and Enhong Chen. Masked video pretraining advances real-world video denoising. IEEE Transactions on Multimedia, 2025.
Shih-Po Lee, Niraj Prakash Kini, Wen-Hsiao Peng. Ching- Ma, and Jenq-Neng Hwang. HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023.
Yadong Li, Dongheng Zhang, Ruixu Geng, Zhi Lu, Zhi Wu, Yang Hu, Qibin Sun, and Yan Chen. A high-resolution handheld wave imaging system with phase error estimation and compensation. Communications Engineering, 3(1):1-11, 2024a. ISSN 2731-3395, doi: 10.1038/ >44172-023-00155--.
Yadong Li, Dongheng Zhang, Ruixu Geng, Jincheng Wu, Yang Hu, Qibin Sun, and Yan Chen. IFNet: Imaging and Focusing Network for handheld mmWave Devices. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024b. doi: 10.1 10910CSSSS4444SSS.. 2024 10444461..
Jiarun Liu, Hao Yang, Hong- Yu Zhou, Yan Xi, Lequan Yu, Cheng Li, Yong Liang, Guangming Shi, Yizhou Yu, Shaoting Zhang, Hairong Zheng, and Shanshan Wang. Swin-UMamba: Mamba-Based UNet with ImageNet-Basse Based Pretraining. In Marius George Linguraru, Qi Dou, Aasa Feragen, Stamatia Giannarou, Ben Glocker, Karim Lekadir, and Julia A. Schnabel (eds.), Medical Image Computing and Computer Assisted Intervention MICCAI 2024, 2024a. ISBN 978-3-031-721 14--4.
Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint arXiv:2401. 10166, 2024b.
Jun Ma. Feifei Li. and Bo Wang. U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arX1v:2401.04722. 2024.
Long Peng, Yang Cao, Yuejin Sun, and Yang Wang. Lightweight adaptive feature de-drifting for compressed image classification. IEEE Transactions on Multimedia, 2024.
Yili Ren, Zi Wang, Yichao Wang, Sheng Tan, Yingying Chen, and Jie Yang. GoPose: 3D Human Pose Estimation Using WiFi. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol, July 2022.
Arindam Sengupta and Siyang Cao. mmPose-NLPP Natural Language Processing Approach to Precise Skeletal Pose Estimation Using mm Wave Radars. IEEE Transactions on Neural Networks and Learning Systems, November 2023.
Arindam Sengupta, Feng Jin, Renyuan Zhang, and Siyang Cao. mm-pose: Real-time human skeletal posture estimation using mmwave radars and cnns. IEEE Sensors Journal, 20017):10032-10044 2020.
Jimmy T. H. Smith, Andrew Warrington, and Scott W. Linderman. Simplified State Space Layers for Sequence Modeling, March 2023.
Ruiyuan Song, Dongheng Zhang, Zhi Wu, Cong Yu, Chunyang Xie, Shuai Yang, Yang Hu, and Yan Chen. Rf-url: unsupervised representation learning for of sensing. In Proceedings of the 28th Annual International Conference Mobile Computing And Networking, pp. 282-295, 2022a.
12

---

Published as a conference paper at ICLR 2025
Yongkun Song, Tian Jin, Yongpeng Dai, and Xiaolong Zhou. Efficient Through- Wall Human Pose Reconstruction Using UWB MIMO Radar. IEEE Antennas and Wireless Propagation Letters, March 2022b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Bing Wang, Changhao Chen, Zhaopeng Cui, Jie Qin, Chris Xiaoxuan Lu, Zhengdi Yu, Peijun Zhao, Zhen Dong, Fan Zhu, Niki Trigoni, et al. P2-net: Joint description and detection of local features for pixel and point matching. In Proceedings of the IEEF/CVF International Conference on Computer Vision. pp. 16004-16013, 2021.
Zhi Wu, Dongheng Zhang, Chunyang Xie, Cong Yu, Jinbo Chen, Yang Hu, and Yan Chen. Rfmask:
A simple baseline for human silhouette segmentation with radio signals. IEEE Transactions on Multimedia, 2022.
Chunyang Xie, Dongheng Zhang, Zhi Wu, Cong Yu, Yang Hu, and Yan Chen. RPM 2.0: RF-based Pose Machines for Multi-Person 3D Pose Estimation. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
Kangwei Yan, Fei Wang, Bo Qian, Han Ding, Jinsong Han, and Xing Wei. Person-in- WiFi 3D: End-to-End Multi-Person 3D Pose Estimation with Wi-Fi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Jianfei Yang, He Huang, Yunjiao Zhou, Xinyan Chen, Yuecong Xu, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, and Lihua Xie. Mm-fi: Multi-modal non-intrusive 4d human dataset for versatile wireless sensing. In seveth- Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL httpps://ppezeeveew net/ forum?ic
1uAsASS1th
Zihan Ye, Shreyank N Gowda, Shiming Chen, Xiaowei Huang. Haotian Xu, Fahad Shahbaz Khan, Yaochu Jin, Kaizhu Huang, and Xiaobo Jin. Zerodiff: Solidified
correlation in zero-shot learning. In The Thirteenth International Conference on Learning Representations, 2025.
URL (tttptt::: httphttss://ppnenettte /ttp::////peeeeieeeeeeee ieneeeeieew.... / forum? i dd 9FRV805
Zhengyou Zhang. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(1 133013133--1334, 2000.
Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018
Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Tor- ralba, and Dina Katabi. Through-wall human pose estimation using radio signals. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7356-7365, 2018a.
Mingmin Zhao, Yonglong Tian, Hang Zhao, Mohammad Abu Alsheikh, Tianhong Li, Rumen Hristov, Zachary Kabelac, Dina Katabi, and Antonio Torralba. basea 3d skeletons. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 267-281, 2018b. 8b.
Zhijie Zheng, Jun Pan, Diankun Zhang, Xiao Liang, Xiaojun Liu, and Guangyou Fang. Through- wall human pose estimation by mutual information maximizing of Things Journal, 2023a.
nets. IEEE Internet
Zhijie Zheng, Diankun Zhang, Xiao Liang, Xiaojun Liu, and Guangyou Fang. Radarformer: End-to- end human perception with through-wall radar and transformers. IEEE Transactions on Neural Networks and Learning Systems, 2023b.
Xiaolong Zhou, Tian Jin, Yongpeng Dai, Yongkun Song, and Zhifeng Qiu. MD-Pose: Human Pose Estimation for Single-Channel UWB Radar. IEEE Transactions Biometrics, Behavior, and Identity Science, October 2023.
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model, February 2024.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
In this appendix, we provide additional details and analyses to supplement our main findings on RFMamba for through-wall human-centric perception. The contents are organized follows:
Section A.1
Evaluation RFMamba's performance in multi-person scenarios, including quantitative results and visual demonstrations.
Section A.2
Detailed analysis of pose estimation performance across different action scenarios, comparing RFMamba with baseline methods.
Qualitative evaluation results, presenting visual comparisons of 3D pose estimation across different methods.
Section A.3
Analysis of action recognition performance using confusion matrices, high- lighting RFMamba's 's capabilities in handling imbalanced data distributions.
Section A.4
Nose Person 1 142.30 137.61
Neck Shoulder Elbow Wrist
Ear Average 173.12 144.49 143.56 167.17 145.46 143.11 157.47
Hip Knee Ankle Eye
149.24
Person 2 146.84 117.41
114.59
122.12 144.50 125.48 125,47 132.27 152.49 118.32 129.71 147.62 177.83 134.99 134.52 149.72 148.98 130.72 143.59
Average 144.57 127.51
131.92
Table 4: Quantitative Evaluation Results of Human Body Joint Reconstruction Error (Unit: MM).
Figure 7: Qualitative Evaluation Results Under Multi-Targer Scenarios.
A.1 MULTI-PERSON SCENARIO EVALUATION
To comprehensively evaluate RFMamba's 's performance in multi-person scenarios, we conducted experiments with both two-person and three-person interactions. We expanded RFMamba's output dimensionality to accommodate different numbers of targets in multi-person prediction.
A.1.1 TWO-PERSON INTERACTION
We first collected dataset of approximately 35,000 pairs of RF frames featuring two- two-person inter- actions. This dataset includes corresponding optical images, action labels, and human keypoints extracted using AlphaPose. Our evaluation focused on challenging scenario where Person 1 per- forms walking while Person executes a sarching-in--lace action. This setup captures complex spatial displacements and joint dynamics.
Table presents the pose reconstruction errors for this two-person scenario. While there slight increase in reconstruction eтror compared to single-person scenarios, RFMamba maintains competitive performance. Person 1 exhibits higher reconstruction errors due to increased motion complexity.
Figure 7 illustrates RFMamba's effective simultaneous reconstruction of two targets. These findings interactions effectively.
demonstrate RFMamba's capability handle basic
A.1.2 THREE-PERSE INTERACTION
To further validate the generalization capability, extended our evaluation to more complex scenarios with three individuals. We collected a comprehensive dataset comprising 48000 pairs of RF frames across different temporal periods, environments, and occlusion conditions (with/without
14

---

Published as a conference paper at ICLR 2025
wooden board obstruction). The dataset includes corresponding optical images, action categories, and human keypoints extracted using AlphaPose. The data was split into a training set and a test set at 4:1 ratio.
The individuals engaged three complex dynamic activities simultancously: walking freely, wav- ing hands, and transitioning between standing and sitting positions. This setup provides more challenging test of the model's capabilities handling multiple targets with diverse motions.
Nose
Neck
Shoulder
Ellow
Wrist
Hip
Knee
Ankle
Eye
Ear
Average
Person 78.56
79.15
111.56
170.25
79.96
81.94
82.31
77.94
79.58
Person 50.49
39.33
40.97
41.13
46.22
36.63
36.14
35.86
49.87
51.37
42.57
Person
86.70
81.95
86.24
100.87
125.49
82.98
85.81
93.32
86.20
87.35
92.51
Average
71.92
66.81
69.67
84.52
113.99
66.52
67.96
70.5
71.34
72.77
76.29
Table reconstruction errors (mm) in three-person scenario without wooden board obstruction.
Nose
Neck
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Eye
Ear
Average
Person 108.60 608.60
94.01
99.90
116.17
140.15
93.66
93.48
102.19
108.76
102.82
106.49
Person
67.06
35.24
38.23
65.89
124.37
31.97
33.23
37.06
68.14
51.54
55.73
Person 96,44
94.37
94.92
96.09
99.82
90.31
81.25
78.33
98.59
99.05
92.66
Average
90.7
74.54
77.68
92.72
121.45
71.98
69.32
72.53
91.83
84.47
84.95
Table 6: Pose reconstruction errors (mm) in three-person scenario with wooden board obstruction.
Tables 5 and 6 present the pose reconstruction errors for three-person scenarios without and with wooden board obstruction, respectively. Despite the increased complexity, RFMamba maintains robust performance across different individuals and occlusion conditions. The variations in error rates be attributed to different motion complexities and occlusion effects.
Figure g demonstrates RFMamba's 's effective simultaneous reconstruction of three targets in both unobstructed and obstructed scenarios. These comprehensive results further support RFMamba's potential for real-world applications involving multiple subjects.
Unobstructed
Optical Image
Ground Truth
Prediction Result
Optical Image
Ground Truth
Prediction Result
Obstructed
Optical Image
Ground Truth
Prediction Result
Optical Image
Ground Truth
Prediction Result
Figure 8: Qualitative results of three-person pose estimation in both unobstructed and obstructed scenarios. For each case: optical image (left), ground truth pose (middle), and RFMamba prediction (right).
A.2 POSE ESTIMATION PERFORMANCE ACROSS DIFFERENT ACTION SCENARIOS
We further provide detailed amalysis of RFMamba's 's pose estimation performance across various tion scenarios, further illustrating its capabilities in diverse conditions. Table pressents comparative analysis of different methods for actions including walking, waving, sitting, and static poses.
15

---

Published as a conference paper at ICLR 2025
Method
Walk
Wave
Sit
Static
Mean
RF-Pose3D
114.28
70.46
65.43
46.46
85.35
mm-Pose
225.69
110.21
141.00
132.88
165.98
ResNet3D-50
150.10
74.91
76.54
74.28
105.34
RadarFormer
443.87
117.09
164.17
86.26
258.90
RFMamba
72.08
34.81
38.97
31.17
50.64
Table 7: Detailed MPJPE (mm) Under Different Action Scenarios.
The results show that RFMamba consistently achieves lower reconstruction errors compared to existing methods across all action categories. In particular, for challenging scenarios such walking, which involve complex non-rigid motions and significant variations in posture and position, RFMamba maintains relatively low reconstruction errors. This performance indicates the model's ability to accurately capture and reconstruct human poses even in dynamic and complex environments.
Imarre
RFManbs
Qualitative comparison of 3D pose estimation results: (a) Optical image for reference, (b) Ground truth pose, and estimated poses by (c) ResNet3D-50, (d) RadarFormer, and (e) RFMamba.
Figure 9:
A.3 QUALITATIVE EVALUATION RESULTS OF DIFFERENT METHODS
Figure 9 provides a visual comparison of 3D pose estimation results from RFMamba and baseline methods to complement the quantitative evaluations. These examples illustrate RFMamba's improved accuracy in pose reconstruction and temporal consistency across frames. While ResNet3D-50 shows overall effectiveness, exhibits notable deviations in extremities. RadarFormer, limited by its simpler network structure and narrow temporal receptive field, struggles with complex poses, only approximating body positions rather than precise joint locations. These visual results underscore RFMamba's ability capture fine- tne-grained spatial information and temporal dynamics.
ResNo
Radar Foemer
60 RFMamba
Figure 10: Comparison of confusion matrices for activity recognition task: (a) ResNet3D-50, (b)
RadarFormer, and (c) RFMamba. Darker colors indicate higher prediction accuracy.
16

---

Published as a conference paper at ICLR 2025
A.4 EVALUATING REID PERFORMANCE WITH CONFUSION MATRIX
Figure presents the average confusion matrices for ResNet3D-50, RadarFormer, and RFMamba, providing insights into their action recognition performance. RFMamba demonstrates superior accuracy across various actions, including those with fewer samples in the long-tailed data distribution.
ResNet3D-50, which infers action classes from single frames, shows bias towards classifying minority samples as the majority class (walking). RadarFormer performs well in distinguishing dynamic actions but struggles with minority classes due insufficient temporal information.
In contrast, RFMamba accurately recognizes both majority and minority classes, showcasing its ability learn effective classification boundaries and its strong generalization capability, even with imbalanced data distributions.
17


Reviews:
[{"review_id": "OzlmYXh523", "reviewer": "Reviewer_uCrw", "summary": "The paper proposes RFMamba, a frequency-aware state space model (SSM) for human-centric perception using RF signals. RFMamba addresses the challenges of processing long-sequence RF data, with an innovative dual-branch SSM block that handles frequency and spatio-temporal cues for better accuracy in human tracking tasks. The model is evaluated on downstream tasks and shows significant performance improvements over existing methods. The proposed method is inspiring and paper is well-structured, but could improve in clarity and visual representation.", "questions": "1) Can the model specifically deal with extreme RF environments in practical use? such as high interference or low signal-to-noise ratios? and what about the demanded resources (data and computation) if adapted to practical scenarios?\n\n2) How does RFMamba's computational efficiency compare to similar SSM or Transformer-based models in terms of real-time performance?\n\n3) How generalizable is RFMamba to other RF-based applications beyond human perception?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The model offers a practical solution to handle long-sequence RF signals with linear complexity. The dual-branch design of the block integrates both frequency and spatio-temporal information, addressing the core challenge for RF signal representation learning. The proposed RFMamba shows clear improvements in localization, recognition, and action detection tasks.", "weaknesses": "1) Some equations and model parameters are not sufficiently explained. This makes it a bit hard to grasp the model's operation and how it ties into the broader methodology.\nFor example, some suggestions for modification:\n- For Figure 1, it would be better to enlarge the font sizes of labels and titles.\n- In Section 3.1.1, after the introduction of frequency and spatial analysis in RF sensing, would be better to connect more to the proposed method and methodology.\n- In Section 3.1.2, eq (4), explain better, not clear what are h and h' meaning, and model parameters A, B, C, D's usage, also can connect more to the overall methodology.\n\n2) While the paper claims linear complexity, it's a bit unclear the resulting computational costs compared with similar models. Some theoretical or empirical comparison would help better clarify its practical deployability.\n\n3) The model's applicability to other domains and signal types can make a straightforward discussion to complete the methodology and usage range.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Kpysq0B0Iq", "reviewer": "Reviewer_oFAp", "summary": "This paper presents a state space model designed for RF-based human perception. The authors introduce various modules, incorporating both frequency domain and spatio-temporal domain components. Additionally, they develop a six-way scanning method within the frequency modeling branch. Experimental results demonstrate the effectiveness of the proposed approach.", "questions": "(1)The authors primarily use a custom MIMO (SFCW) radar to validate their proposed method. Consequently, the paper’s title, which references RF-based human-centric perception, may overstate the scope, as other radar types and RF-based sensing modalities (e.g., Wi-Fi, RFID, LoRa) are not examined. The authors should clarify their motivation for selecting SFCW radar, including its advantages over existing options like TI mmWave radar.\n\n(2) The introduction of SFCW radar in Section 3.1.1 lacks clarity. For example, it is unclear how Eq. 3 represents the signal across three key dimensions: fast time (distance), antenna (azimuth and elevation), and slow time (velocity across echoes). A more detailed explanation in this section would improve understanding.\n\n(3) In the related work section, more state-of-the-art studies on wireless signal sensing relevant to RF pose estimation should be reviewed. The authors should also discuss distinctions between the proposed Mamba method and other Mamba-based methods.\n\n(4) For the framework depicted in Fig. 2, while each block is briefly explained, more specific functionality should be provided, including details on the SiLU block. Additionally, the loss function and training approach require elaboration. For instance, the rationale for using different task losses to define the joint training loss in Eq. 11 should be clarified. Since the code is not provided, a detailed description of the training algorithm is needed.\n\n(5) Though the proposed method is tested with a real-world setup, it would benefit from validation using a public dataset, such as an mmWave radar dataset. The authors should also consider benchmarking against advanced transformer models and other Mamba methods. Furthermore, the evaluation on generalization and multi-person scenarios is limited; additional tests across different time (different days) and involving more than three individuals would strengthen the findings.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "(1) This paper is the first to apply the Mamba method for RF-based sensing, marking a novel contribution to the field.\n\n(2) The authors provide comprehensive discussions for each module in the proposed framework.\n\n(3) A six-way scanning strategy in the frequency modeling branch is proposed, enabling adaptive selection of the most informative frequency cues.\n\n(4) Real-world experimental results demonstrate the effectiveness of the proposed method in terms of both accuracy and parameter efficiency.", "weaknesses": "(1) The introduction sections on RF sensing and SFCW are underdeveloped. A more detailed discussion on the model and data specifics for SFCW radar is needed.\n\n(2) The related work section is limited. The authors should include a review of state-of-the-art wireless signal sensing work relevant to RF pose estimation and clarify distinctions between the proposed model and other Mamba-based approaches.\n\n(3) The framework of the model requires a more detailed discussion, and the training algorithm should be included in the paper.\n\n(4) The experimental results could benefit from further enhancement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JeWNRKDKqJ", "reviewer": "Reviewer_5Q2S", "summary": "This paper proposes a State Space Model (SSM) based method for human-centric perception with radio frequency (RF) signals. Specifically, the authors devise a two-branch model to learn frequency and spatial-temporal features, respectively. The authors have also conducted comprehensive experiments to demonstrate the superiority of their method.", "questions": "1. Can the authors provide more discussions about cross-domain sensing?\n2. Can the authors compare their methods in terms of detailed efficiency?\n3. Can the authors discuss more related works about wireless sensing?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of Mamba models in RF-based sensing is novel.\n2. The authors have collected a new dataset and evaluated their method with three downstream tasks.", "weaknesses": "1. The motivation of using Mamba models is not clear enough. The authors mention that Mamba models are more capable of tackling long sequences, while the authors have not discussed the performance of models in terms of different lengths of sequences.\n2. A common challenge in RF-based sensing is cross-domain sensing, since RF signals may differ between different environments. I understand that this paper pioneers the use of Mamba models for RF-based sensing, rather than focusing on cross-domain sensing, but the authors should provide more discussions on this.\n3. The authors have compared their method with baselines in terms of the number of parameters, while the authors can also compare their detailed efficiency, such as training time, testing time, and throughput.\n4. Many related works about Wireless Signal Sensing have not been discussed. For example, recent works have involved the use of WiFi signals, ultra-wideband radars, millimeter wave radars. The authors only discussed three related papers about wireless sensing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRRtQUDtph", "reviewer": "Reviewer_aiy3", "summary": "The paper presents RFMamba, a novel frequency-aware state space model for RF-based human-centric perception (HCP), designed to process radio frequency (RF) signals for tasks like human pose estimation, activity recognition, and person re-identification (ReID). Leveraging the capabilities of state space models (SSMs), RFMamba uses a dual-branch architecture that combines frequency and spatiotemporal modeling to effectively capture the characteristics of RF signals. The model introduces an omni-dimensional scanning mechanism, selectively focusing on informative frequency cues. Experiments on the newly introduced THP dataset (covering both free-space and wall-occlusion scenarios) demonstrate RFMamba’s superior performance over existing methods in accuracy.", "questions": "1. What are the specific activities in the THP dataset, and what is their distribution?\n2. Were participants stationary or moving freely during activities?\n3. Would the model generalize to other environments without additional data collection or fine-tuning?\n4. The authors used SFCW radars for evaluation. Would the proposed method generalize to FMCW radar systems as well, and if so, what adjustments might be necessary?\n5. Will the THP dataset be made publicly available to support reproducibility and further research in RF-based human perception?\n6. What are the compute and memory requirements for achieving real-time performance with RFMamba?\n7. The dataset was split into a 4:1 training-testing ratio. How much data was allocated for validation, and what criteria were used for this split?\n8. For pose estimation, the authors used the metric Mean Per Joint Position Error (MPJPE). Is this measured in 3D or 2D space?\n9. Was IRB approval obtained for the human subject data collection, and were participants informed and provided consent for the use of their radar and camera data in this study?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Motivation and practical use cases: The authors address real-world challenges in RF-based human perception, aiming to support applications like security, health monitoring, and emergency response by enabling through-wall sensing and multi-person tracking where visual methods may be ineffective.\n\n2. Technical novelty with Mamba modules: RFMamba leverages Mamba-based state space modeling specifically adapted for RF signals, incorporating frequency and spatiotemporal Mamba modules to capture long-term dependencies within complex RF data.\n\n3. Generally clear presentation: The paper is well-structured and understandable, though some details are missing, particularly regarding some design choices, data collection and experimental setup (see weaknesses below).", "weaknesses": "1. Lack of sequence length analysis: While the authors emphasize the model's capability to handle variable and long input sequences, they do not provide an analysis of how sequence length affects performance, efficiency, or stability. It would be useful to see experiments with varying sequence lengths, insights into computational complexity and memory impact, and clarification on the sequence length used in their experiments.\n2. Potential for improved transformer-based model: The lower performance of RadarFormer likely reflects specific design choices rather than a fundamental transformer limitation for RF perception (e.g., using only amplitudes, using two successive frames instead of 12). The authors did not explore enhancements to RadarFormer, such as incorporating phase information, extending the temporal context, or adding RF-specific adaptations, which may have yielded results comparable to or better than RFMamba, questioning the necessity of Mamba-based SSMs.\n3. Unclear design choices: Some design decisions are not fully explained. For instance, in Figure 3(a), the scanning path appears to follow a bidirectional approach converging at the center. It is unclear why the authors did not consider capturing both sequences (from T−6 to T+6 and T+6 to T−6) and concatenating them, which could potentially enrich the temporal representation and improve performance.\n4. Lack of details on activity types, distribution, and environment diversity: The paper lacks specifics on the 19 activities in the THP dataset, their frequency distribution, and whether participants performed these activities statically or while moving freely. This is important for understanding the model’s generalization across activity types and movement patterns in RF-based perception. Additionally, the dataset was collected primarily in a single environment (with and without a wall), limiting environmental diversity. It remains unclear if the model would perform similarly in different settings, whether it would need extensive data collection, or if fine-tuning would be required in new environments.\n5. Limited dataset accessibility: Although the THP dataset provides valuable scenarios for evaluating RF-based human perception, it is not confirmed for public release, restricting reproducibility and broader research applications in this area.\n6. Unclear real-time performance: The paper lacks analysis on the compute and memory requirements needed for real-time performance, leaving practical deployment feasibility uncertain.\n7. Lack of generalization comparison with baseline models: While the authors claim that RFMamba generalizes well across various scenarios, there is no direct comparison with existing methods on generalization performance. Without showing how baseline models perform in similar conditions, it is difficult to assess whether RFMamba’s improvements are specific to the training environment or if they extend to varied, unseen scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)