# mmHSE: A Two-Stage Framework for Human Skeleton Estimation Using mmWave FMCW Radar Signals
###### Abstract
The second approach adopts heatmap-based methods [20; 21; 22; 23], where time-frequency representations--such as range-angle (RA) maps--are derived from intermediate frequency signals and fed into deep neural networks [24]. These representations preserve richer motion and frequency-domain information, offering more informative features compared to sparse point clouds. Despite their advantages, several critical limitations remain in existing work:  
* Limited model generalization: Most existing methods are optimized for controlled environments, with limited exploration to cross-domain or cross-user robustness. This restricts their applicability in complex and dynamic real-world settings. * Lack of physiological constraints: Current approaches rarely incorporate human anatomical priors or kinematic constraints, often leading to pose estimates that violate basic biomechanical feasibility and deviate from natural human motion patterns. To address the technical limitations of existing mmWave FMCW radar-based HSE systems, we propose a novel two stage dual-radar framework, termed mmHSE, which employs orthogonal radar deployment to enhance spatial coverage and capture complementary cross-view information. To support model development and evaluation, we construct a middle-scale dataset comprising 30,000 raw Analog-to-Digital Converter (ADC) sequences collected from 12 participants across three representative indoor scenarios: a meeting room, a hallway, and an open space. A custom signal processing pipeline is developed to transform raw intermediate frequency (IF) signals into high-resolution RA maps; we adopt a two-stage estimation architecture that jointly extracts spatiotemporal features and refinesskeletal topology. In Stage 1, lightweight depthwise separable convolutional networks, enhanced with self-attention modules, independently extract multi-scale spatiotemporal features from dual-radar inputs. These complementary features are fused via a cross-branch attention mechanism to produce an initial estimate of 21 skeletal keypoints. In Stage 2, we introduce a skeletal topology optimizer based on graph convolutional networks (GCNs), which incorporates biomechanical priors--such as bone length consistency and joint angle constraints--to refine the predicted skeleton into anatomically plausible configurations. We extensively evaluate the performance of the proposed mmHSE system through a series of experiments. To assess the effectiveness of RA maps for human skeleton estimation, we benchmark several representative deep learning architectures, including ResNet-18 [25], Long Short-Term Memory (LSTM) [26], Convolutional Neural Networks combined with LSTM (CNN-LSTM) [27], and Mobile-friendly Vision Transformer (MobileViT) [28]. Experimental results show that these baseline models achieve estimation errors within 8 cm, confirming the viability of RA maps as informative representations for radar-based HSE. Building on this foundation, we further reduce the average error to 2.67 cm, significantly outperforming existing methods and validating the effectiveness of the dual-radar configuration and anatomical refinement module. To evaluate generalization, we employ a stratified dataset partitioning scheme to test the system under cross-domain and cross-user conditions. The mmHSE framework consistently maintains an average error below 4 cm in both scenarios, demonstrating strong robustness to environmental variability and inter-user differences. Beyond accuracy benchmarking, we also conduct comprehensive analyses of critical performance factors, including keypoint-wise localization errors, ablation studies of functional modules, and the impact of historical input sequence length. The primary contributions of this paper are as follows:  
* We propose a novel two-stage human skeleton estimation framework mmHSE, based on mmWave FMCW radar, covering the entire process from data acquisition to learning-based skeletal estimation, and featuring a modular dual-stage design tailored for radar-based sensing.