Scalable RF Simulation in Generative 4D Worlds

## Abstract

Abstract Radio Frequency (RF) sensing has emerged as a powerful, privacy-preserving alternative to vision-based methods for indoor perception tasks. However, collecting high-quality RF data in dynamic and diverse indoor environments remains a major challenge. To address this, we introduce WaveVerse , a prompt-based, scalable framework that simulates realistic RF signals from generated indoor scenes with human motions. WaveVerse introduces a language-guided 4D world generator, which includes a state-aware causal transformer for human motion generation conditioned on spatial constraints and texts, and a phase-coherent ray tracing simulator that enables the simulation of accurate and coherent RF signals.
Experiments demonstrate the effectiveness of our approach in conditioned human motion generation and highlight how phase coherence is applied to beamforming and respiration monitoring. We further present two case studies in ML-based high-resolution imaging and human activity recognition, demonstrating that WaveVerse not only enables data generation for RF imaging for the first time, but also consistently achieves performance gain in both data-limited and data-adequate scenarios.

## 1 Introduction

Radio frequency (RF) sensing has emerged as a compelling modality for tasks such as 3D imaging, human activity recognition, and health monitoring [24, 35, 61].
In safety-critical or low-visibility scenarios, RF-based methods [24, 36, 37] offer high-resolution imaging despite fog, smoke, or occlusion.
At the same time, RF sensors do not capture images or videos, making them inherently privacy-preserving and well-suited for contactless and continuous health monitoring, including vital sign monitoring [13, 59, 18], sleep analysis [60, 15], and mental health assessment [26, 14].
Despite these advantages, acquiring large-scale and high-quality RF sensing datasets remains challenging. Building such datasets requires capturing a wide range of room layouts, human activities, and individual differences, all of which demand significant cost and effort. Worse still, RF sensing systems differ widely in hardware configurations (i.e., bandwidth, antenna layout, and signal modulation), making it difficult to share or reuse data across systems.
Unlike vision or audio, RF sensing lacks standardized and unified datasets, which limits generalization across systems and slows research progress.

Recent efforts have explored both physics-based simulation [57, 3] and learning-based synthesis [8, 9] to address these challenges.
However, existing approaches focus on signal interactions with human bodies while neglecting the surrounding environment.
This is particularly problematic for RF sensing, where multipath propagation (i.e., multi-bounce reflections between the human body and surrounding structures such as walls, floors, and objects) significantly affects the received signal and is a key factor limiting generalization [53, 43].
Moreover, learning-based synthesis [8, 9] still requires a training dataset to begin with and remains limited to a specific sensor configuration.

In this paper, we introduce WaveVerse, a hybrid generation–simulation framework for synthesizing realistic and diverse RF signals.
As illustrated in Fig. [1](#S1.F1), WaveVerse combines 4D world generation with physics-based RF simulation.
Specifically, it leverages the emergent capabilities of Large Language Models (LLMs) [1, 19], shown in various tasks [6, 38], to generate diverse 3D indoor environments and dynamic human motions within them.
Given the 4D world (i.e., a 3D environments with dynamic human motions), WaveVerse employs a ray tracing engine that accurately models multipath propagation and provides consistent signals across antennas and over time. This hybrid design combines the best of both worlds: generative diversity from 4D synthesis and physical realism from RF simulation.
The use of explicit mesh representations for 3D layouts provides additional benefits. It enables aligned supervision for RF learning tasks (e.g., depth estimation, semantic segmentation, human poses) and supports RF simulation with flexible sensor configurations for a wide range of downstream applications.

Figure: Figure 1: Overview of WaveVerse. Our system first generates 4D worlds with moving humans in 3D scenes. It then simulates RF signals with the proposed phase-coherent ray tracing.
Refer to caption: /html/2508.12176/assets/x1.png

WaveVerse introduces two key innovations to achieve these new capabilities.
In scene synthesis, we develop a state-aware causal transformer to generate human motion conditioned on both language description of the motion and spatial constraints within the environment, producing dynamic humans that are physically plausible and contextually consistent with the environment.
We also introduce a phase-coherent ray tracing mechanism that enables accurate and consistent simulation of signal phase, which is crucial for RF imaging and sensing tasks that rely on phase in addition to signal amplitude.

We evaluate the effectiveness of WaveVerse through extensive experiments.
In microbenchmark experiments, WaveVerse effectively generates diverse 3D indoor environments with dynamic humans.
Our proposed state-aware causal transformer outperforms baselines including diffusion-based ones in human motion generation.
Second, we assess the impact of phase coherence by comparing our phase-coherent ray tracing engine with conventional ones, and observe significant improvements in both circular beamforming imaging and respiration monitoring tasks.
Finally, we conduct two case studies on high-resolution RF imaging and human activity recognition. We show that WaveVerse enables data simulation for RF imaging tasks for the first time and achieves consistent performance improvements in both data-limited and data-adequate settings.

In summary, our contributions are: (1) We introduce WaveVerse, a prompt-based, scalable framework for realistic RF signal simulation in the 4D indoor world. (2) We develop a state-aware causal transformer that enables human motion generation conditioned on language and paths. (3) We propose phase-coherent ray tracing that enables RF simulation for phase-sensitive RF applications. (4) We demonstrate the effectiveness of WaveVerse through extensive experiments and case studies, and will release our [code](https://zhiwei-zzz.github.io/WaveVerse/) and [simulator](https://zhiwei-zzz.github.io/WaveVerse/) to facilitate future research.

## 2 Related Work

RF Simulation.
Ray tracing has been widely used for radio propagation modeling, particularly in static environments [17, 51, 52]. For dynamic objects, [49, 57] connect object meshes to antennas but ignore environment factors, making them unsuitable for large indoor scenes. Inspired by advances in image generation [10, 16, 23], recent works have explored data-driven alternatives. RF-Genesis [8] combines ray tracing of moving humans with network-generated signals conditioned on a single-view environment image, while RF-Diffusion [9] leverages language-conditioned diffusion models with limited controllability. Moreover, they require large annotated datasets, lack physical interpretability, and struggle to model key RF properties like multipath effects and phase coherence.
While HFSS [5] supports RF signal simulation by solving Maxwell’s equations or leveraging physical optics, they involve significant computation and are impractical for simulating large-scale, dynamic indoor scenes.

Conditional Human Motion Generation.
Conditional human motion generation can be generally split into two categories based on model structures. For text-conditioned motion generation, [12, 21, 54, 58] employ autoregressive models, while others use VAE [11] or diffusion models [7, 34, 39, 56] to generate motions at a sequence level. Recently, text and trajectory-conditioned generation has received increasing attention, where joint positions at specific frames are provided as additional control. MDM [39] and PriorMDM [34] treat this as a motion inpainting task. OmniControl [48] builds on ControlNet [55] and applies analytic functions for trajectory refinement during denoising, while TLControl [42] frames it as a regression problem with test-time optimization, achieving state-of-the-art results. Notably, all of these methods generate motion at the sequence level. However, trajectory creation demands carefully designed sequences with smooth, predefined velocities that align with texts, resulting in substantial manual effort. In addition, these methods require the motion length to be known in advance. While some early work has focused on scene-aware motion generation [4, 45] with sequence-level models following a guidance-then-generation paradigm, these approaches are limited by short motion durations with datasets [44], averaging less than two seconds.

## 3 Method

WaveVerse is an automated LLM-powered framework for simulating realistic RF signals in 3D indoor environments with human motions.
As a prompt-driven framework, WaveVerse can be used either interactively by a human user or fully automated by an LLM agent.
Given a text description of an indoor environment, WaveVerse generates a text-aligned indoor environment with dynamic human activity, and finally simulates RF signals of the scene.
This section describes the two core components of WaveVerse: (1) the 4D scene generator that synthesizes diverse indoor 4D scenes (Sec. [3.1](#S3.SS1)), and (2) a phase-coherent ray tracing engine for signal simulation (Sec. [3.2](#S3.SS2)).

### 3.1 4D World Generation

Figure: Figure 2: Overview of 4D World Generation.
Refer to caption: /html/2508.12176/assets/fig/method-generation-v3.jpg

WaveVerse utilizes a prompt-based pipeline to enable fully automated 4D world generation. Given a text description of the desired environment – whether provided by a user or an LLM – WaveVerse first constructs a semantically aligned 3D environment along with corresponding human body shapes. It then synthesizes dynamic human motion within the environment. To support realistic and physics-based RF simulation (Sec. [3.2](#S3.SS2)), WaveVerse also assigns dielectric properties to scene objects using LLMs. We describe each component of the generation process in detail below.

3D Environment and Human Shape Generation.
WaveVerse begins with a text description of the environment (Fig. [2](#S3.F2)).
We build on an existing generation pipeline [50] to produce a structured layout, including floor plans, object categories, and placements, ultimately yielding a mesh representation of the indoor environment.
This explicit 3D representation serves as a foundation for simulating RF signals as well as other modalities such as RGB images and depth maps.
For human modeling, WaveVerse uses the SMPL model [27], a parametric human mesh that can be animated by adjusting pose and shape parameters.
These shape parameters can be manually specified or automatically generated using a finetuned LLM [2], which is conditioned on plausible body descriptions inferred from the input environment text by a general-purpose LLM.

Motion Description and Path Generation.
To enable scalable motion generation for RF-based applications, WaveVerse animates SMPL with sequences of generated joint positions, which we refer to as motion throughout this paper.
The core challenge lies in generating human motion that aligns with the semantic context of the environment, while maintaining diversity and spatial realism, like not clipping through walls.
One approach to achieve this level of control is to provide a motion description along with *dense trajectory* information, that is, precise 3D joint positions or velocities of key joints, aligned with the text description and the environment.
However, this approach requires significant manual effort, as specifying the trajectory effectively constitutes generating the motion itself, predefining speed, balance, and action, and ultimately limits scalability.

To overcome this challenge, we decompose motion generation into two steps. First, given a prompt that textually encodes the environment, an LLM generates a textual description of the desired motion, such as “wave the arm” or “walk and almost slip,” and specifies only the start and end positions of the motion, which can also be provided explicitly by users if needed.
We then relax the trajectory condition with a path condition: a *set* of spatial waypoints that guide where the person might be.
This path can be automatically generated using agent-defined start/end points based on the 3D environment and a path-finding algorithm.
We delegate the complex task of generating exact joint positions to the downstream motion generation model, while leveraging the strength of LLMs to reason about general spatial context and motion diversity.

Conditional Human Motion Generation. The second step is to generate motion sequences conditioned on the obtained text and path inputs. Since the path does not specify motion duration, we adopt an autoregressive model that dynamically determines when to terminate the sequence, unlike existing methods that generate human motion at the sequence level with a pre-specified length  [39, 47].
Specifically, motion sequences are first tokenized using VQ-VAE [40], achieving motion tokens $X=[m_{1},m_{2},\ldots,m_{n},m_{\text{end}}]$, where $m_{i}\in\{1,\ldots,M\}$ indexes a learned codebook, and $m_{\text{end}}$ denotes the end of the sequence.
The motion description is encoded using CLIP [32], while the 2D waypoint sequence is processed through an MLP-based position encoder, producing condition embeddings $c=(c_{\text{text}},c_{{\text{path}_{0}}},\ldots,c_{{\text{path}_{L}}})$.

Figure: Figure 3: State-Aware Causal Transformers.
Refer to caption: /html/2508.12176/assets/x2.png

While existing autoregressive models [21, 54] formulate motion generation as next-token prediction, learning the distribution $P(m_{n}\mid c,m_{0},\ldots,m_{n-1})$, we observe that this formulation struggles to align the generated motion with the input path.
Inspired by reinforcement learning [22, 33, 46], we consider next-token prediction as a sequential decision-making process, where each generated motion token is treated as an action.
We argue that the absence of explicit spatial context at each decision step limits the model’s ability to follow the intended path.
To address this, we introduce a state-aware causal transformer, as shown in Fig. [3](#S3.F3), which conditions each token prediction on the current spatial state of the human.
Formally, the next-token distribution is modeled as $P(m_{n}\mid c,m_{0},s_{0},\ldots,m_{n-1},s_{n-1})$, where $s_{i}$ encodes the 2D position of the human at the final frame of the motion reconstructed up to token $m_{i}$, using the same position encoder.

Despite the benefits of spatial state conditioning, the model overfits by relying heavily on path information, resulting in poor alignment with the motion descriptions. To mitigate this and promote balanced conditioning on both text and path, we introduce a path-masking strategy during training.
Specifically, we sample a masking ratio from the range $[r_{\min},r_{\max}]$ to determine the number of waypoints to mask.
To apply this masking, we iteratively select and mask random contiguous segments of length $\ell$.
If further masking is needed and no full segments remain, we continue by randomly masking individual waypoints until the target ratio is reached.
We find that this sequential masking strategy improves generalization and enhances text-motion alignment (Sec. [4.1](#S4.SS1)).

Dielectric Property Generation.
To further enhance physical realism, WaveVerse supports prompt-driven assignment of dielectric properties to scene objects.
Given object descriptions, an LLM generates semantically meaningful dielectric parameters including permittivity and conductivity.
This mechanism provides fine-grained control over RF signal behavior in a fully automated manner, improving both the scalability and realism of RF simulation in WaveVerse.

### 3.2 RF Signal Simulation

Given the generated 4D scenes, WaveVerse employs ray tracing to simulate RF signals.
However, conventional graphics-oriented ray tracing engines [29, 30] primarily model signal amplitude and cast rays stochastically within each pixel.
When adapted to RF simulation, rays are typically cast randomly over a spherical or conical distribution, resulting in inconsistent ray-surface interactions across frames and radar positions.
While suitable for visual rendering, this stochasticity poses significant challenges when extended to RF applications, where signal phase plays a critical role.
For example, high-resolution RF imaging distinguishes objects at the same range but different angles by leveraging phase differences through beamforming.
Similarly, Doppler-based velocity estimation relies on phase shifts across consecutive frames caused by object motion.
To address this, we introduce *phase-coherent ray tracing* that operates on the scene mesh and ensures consistent ray-surface interactions across different radar positions and over time as objects move.
This preserves signal phase coherence, enabling accurate simulation of phase-dependent RF phenomena.

Figure: Figure 4: Illustration of phase-coherent ray tracing. (a) shows spatial coherence by tracing consistent paths across two radar locations. (b) depicts temporally coherent ray tracing for a moving person across timestamps $t_{1}$, $t_{2}$, and $t_{3}$. For clarity, the rays at $t_{2}$ and $t_{3}$ are omitted.
Refer to caption: /html/2508.12176/assets/fig/method-raytracing-v3.jpg

RF Simulation with Ray Models.
Ray tracing models wave propagation as a collection of discrete paths connecting the transmitter (Tx) and receiver (Rx) through the environment.
Let $\{\mathcal{P}_{k}\}$ where $k=1,\ldots,K$ denote the set of valid propagation paths identified by ray tracing.
Each path $\{\mathcal{P}_{k}\}$ is characterized by four parameters:
the propagation delay $\tau_{k}$;
a complex coefficient $a_{k}$, whose magnitude encodes attenuation due to path loss and interactions with scene surfaces, and whose angle represents accumulated phase shifts;
the angle of departure (AoD) $\theta_{k}$ at Tx;
and the angle of arrival (AoA) $\varphi_{k}$ at Rx.
The channel impulse response (CIR) $h(t)$, which describes how an impulse propagates from Tx to Rx, is modeled as the superposition of all paths:
$h(t)=\sum_{k}a_{k}\cdot G_{\text{Tx}}(\theta_{k})\cdot G_{\text{Rx}}(\varphi_{k})\cdot\delta(t-\tau_{k}),$
where $G_{\text{Tx}}$ and $G_{\text{Rx}}$ denote the antenna gain patterns of the transmitter and receiver, capturing their directionality, and $\delta(t)$ is the Dirac delta function.
Any signal received by Rx can then be computed as the convolution between the transmitted signal and the CIR.

Phase-Coherent Ray Tracing.
As mentioned above, conventional ray tracing methods fall short in preserving phase coherence, as they cast rays stochastically, resulting in different ray-surface interactions even for nearby radar positions (Fig. [4](#S3.F4)(a), left).
This issue becomes more severe in dynamic scenes with moving humans, where changes in geometry cause rays to strike entirely different surface points across frames, breaking temporal phase coherence shown in Fig. [4](#S3.F4)(b).

To overcome these challenges, we propose phase-coherent ray tracing that ensures consistent ray-surface interactions across both space and time. To achieve *spatially-coherent ray tracing*, i.e., ensuring that phase varies coherently across different radar locations, we generate paths at each radar location based on a fixed set of paths traced from a representative reference radar.
Specifically, assume we are synthesizing signals for $N$ radars with poses $\{(\mathbf{t}_{n},\mathbf{r}_{n})\}$ for $n=1,\ldots,N$, where $\mathbf{t}_{n}$ and $\mathbf{r}_{n}$ denote the 6D pose, position and orientation, of the transmitter and receiver.
We define a reference $(\mathbf{t}_{0},\mathbf{r}_{0})$ as the geometric center of all radar positions, and trace rays uniformly over a sphere, which achieves a set of paths $\{\mathcal{P}_{k}\}$ between $\mathbf{t}_{0}$ and $\mathbf{r}_{0}$.
For each path $\mathcal{P}_{k}$, we represent it with a sequence of 3D points $\mathcal{P}_{k}=[\mathbf{t}_{0},\mathbf{p}_{0},\cdots,\mathbf{p}_{D_{k}},\mathbf{r_{0}}]$ where $\mathbf{p}_{d}$ denotes the $d$-th surface interaction point along the path and $D_{k}$ denotes the number of surfaces the path interacts with.

To generate paths for a radar with poses $(\mathbf{t}_{n},\mathbf{r}_{n})$, we modify each reference path $\{\mathcal{P}_{k}\}$ by replacing the original transmitter and receiver positions with the new ones, as shown in the right side of Fig. [4](#S3.F4)(a).
We then compute the CIR for each modified path using updated propagation delay, attenuation, phase, AoD, and AoA.
Occlusion checks are further performed on the achieved paths, and any blocked paths are discarded.
By preserving consistent surface interaction points, our approach ensures spatial phase coherence across radars with various poses while avoiding redundant ray tracing.

To obtain *temporally-coherent ray tracing*, i.e., coherent phase changes as humans move within the scene, we remap ray-surface interactions from individual vertices to semantically or spatially coherent groups.
While ray tracing is performed independently over time, we enable temporal coherence by expanding ray hits over stable vertex groups that persist across frames.

Specifically, we partition all vertices $\mathcal{V}=\{\mathbf{v}_{m}\}_{m=1}^{M}$ of a human mesh into $G$ disjoint, semantically coherent groups $\{\mathcal{V}_{g}\}_{g=1}^{G}$, where $\cup_{g}\mathcal{V}_{g}=\mathcal{V}$, with a grouping function $\mathcal{G}:\mathbf{v}_{m}\mapsto\{1,\ldots,G\}$.
At each timestamp $t$, ray tracing yields a set of paths $\{\mathcal{P}_{k}^{(t)}\}$.
When a path $\mathcal{P}_{k}^{(t)}$ intersects the human mesh at point $\mathbf{p}_{d}^{(t)}$, we associate it with a representative vertex $\hat{\mathbf{p}}_{d}^{(t)}$, a fixed vertex of the intersected face, to enable consistent grouping.
We then expand the path by replacing the hit point with all vertices within the same group, i.e., those satisfying $\mathcal{G}(\mathbf{v}_{m})=\mathcal{G}(\hat{\mathbf{p}}_{d}^{(t)})$.
More specifically, the following set of paths
$\left\{\left[\mathbf{t},\ldots,\mathbf{p}^{(t)}_{d-1},\mathbf{v}_{m},\mathbf{p}^{(t)}_{d+1},\ldots,\mathbf{r}\right]\ \middle|\ \mathcal{G}(\mathbf{v}_{m})=\mathcal{G}\left(\hat{\mathbf{p}}_{d}^{(t)}\right)\text{ for }m\in\{1,\ldots,M\}\right\}$
are created to replace $\mathcal{P}_{k}^{(t)}$. We perform occlusion checks on the expanded paths and denote the number of valid paths as $N_{\text{valid}}$.
For each valid path, we compute the propagation delay, attenuation, phase, AoD and AoA. To preserve the overall signal energy, the attenuation is further divided by $N_{\text{valid}}$.
In practice, we apply this expansion only to the first hit point from the transmitter, both to avoid exponential path growth from higher-order reflections and because single-bounce paths typically dominate received energy due to lower propagation loss.

Flexible Configuration.
Since the CIR depends solely on scene geometry and dielectric properties, WaveVerse generalizes to a wide range of radar configurations, including arbitrary antenna positions and orientations, gain patterns, frequency bands, and sampling rates, making it broadly adaptable to diverse hardware setups.
It simulates the received signal for any transmitted waveform via convolution, supporting diverse RF protocols.
Since the simulation is based on explicit physical modeling, WaveVerse scales to unseen conditions while achieving accurate and reliable signal behavior, offering robustness and scalability that are difficult to achieve with data-driven methods.

## 4 Experiments

In this section, we evaluate the 4D world generation and the signal simulation in WaveVerse.
We begin with benchmarks and ablation studies of the proposed state-aware causal transformer for text- and path-conditioned motion generation and analyze the generated 4D world.
We then assess the benefits of phase-coherent ray tracing over existing baselines on three phase-sensitive RF tasks.
Finally, we demonstrate the utility of WaveVerse in two real-world case studies, showing that the generated data improves model performance in both limited and adequate data conditions.

### 4.1 Performance of Human Motion Generation

Dataset.
We evaluate models over the HumanML3D [11] dataset for performance benchmarks.
It contains 14,146 captioned human motion sequences.
We downsample and project the trajectory of the pelvis to $L=64$ 2D waypoints with even distances as the path condition.

Evaluation Metrics.
Following the evaluation protocol from OmniControl [47], we report R-Precision to quantify the alignment between the text and the motion, the Frechet Inception Distance (FID) to assess motion quality, and the Diversity score to measure motion variability.
To assess spatial alignment with the path condition, we define the Path Error as the average per-point $L_{2}$-distance between the generated path and the ground-truth path, and the Ending Error as the deviation at the last timestamp.
For both errors, we use the mean and the percentage of samples exceeding thresholds of 20 and 60 cm to characterize the distribution.

Comparison to Baselines.
We adopt three open-sourced, state-of-the-art motion generation methods to serve as baselines: the diffusion-based methods, MDM [39] and OmniControl [47], and the autoregressive model T2M-GPT [54]. Details on how we adapt all these baselines to incorporate path-based conditioning are provided in the Supplementary Material.

As shown in Tab. [1](#S4.T1), our method consistently outperforms all baselines across R-Precision, FID, and path-following metrics, demonstrating better motion quality and alignment with input conditions.
MDM approaches trajectory-conditioned motion generation as an inpainting task.
However, this formulation assumes known keyframes and lacks motion planning or spatial feedback, limiting its ability to follow global path constraints in our setting.
For OmniControl, its original per-frame joint control signals are replaced with global path conditions, removing localized frame-wise guidance.
Additionally, the path in the analytic function is computed as a weighted sum over joint positions, where the weights vary across denoising steps, leading to gradient instability and increased sensitivity to learning rates.
As a result, it exhibits a higher proportion of large path-following errors, despite maintaining competitive accuracy at lower thresholds.
In contrast, our method operates in a stable, end-to-end autoregressive manner with spatial state feedback and does not require the motion length to be predefined, leading to more precise and controllable motion generation.

**Table 1: Text- and path-conditioned motion generation performance. Bold for the best and underline for the second best. R-Prec.: R-Precision; Div.: Diversity.**
| Method | Architecture | R-Prec. $\uparrow$ | FID $\downarrow$ | Div. $\rightarrow$ | Path Error $\downarrow$ | Ending Error $\downarrow$ |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| $>20$ cm | $>60$ cm | $>20$ cm | $>60$ cm |  |  |  |  |  |
| Ground Truth |  | 0.797 | 0.002 | 9.503 | 0. | 0. | 0. | 0. |
| MDM [39] | Diffusion | 0.719 | 0.295 | 9.462 | 0.547 | 0.207 | 0.666 | 0.367 |
| OmniControl [47] | Diffusion | 0.751 | 0.319 | 9.279 | 0.239 | 0.083 | 0.330 | 0.152 |
| T2M-GPT [54] | Autoregressive | 0.691 | 0.377 | 9.736 | 0.406 | 0.127 | 0.545 | 0.255 |
| Ours | Autoregressive | 0.755 | 0.238 | 9.445 | 0.208 | 0.045 | 0.325 | 0.111 |

**Table 2: Ablation study for key components and hyperparameters in our model.**
| Setting | Model / Variant | R-Precision $\uparrow$ | FID $\downarrow$ | Mean Path Error $\downarrow$ | Mean Ending Error $\downarrow$ |
| --- | --- | --- | --- | --- | --- |
| Components | Ours | 0.755 | 0.238 | 0.151 | 0.287 |
| w/o Mask | 0.643 | 0.747 | 0.192 | 0.325 |  |
| w/o Mask & State | 0.691 | 0.377 | 0.274 | 0.528 |  |
| Masking Rate | [0.5, 0.9] | 0.755 | 0.238 | 0.151 | 0.287 |
| [0.1, 0.5] | 0.691 | 0.396 | 0.171 | 0.312 |  |
| [0.1, 0.9] | 0.713 | 0.298 | 0.160 | 0.303 |  |
| Segment Length | 5 Points | 0.755 | 0.238 | 0.151 | 0.287 |
| 10 Points | 0.763 | 0.342 | 0.207 | 0.393 |  |
| 15 Points | 0.776 | 0.339 | 0.228 | 0.403 |  |

Ablation Studies. We validate key designs of the state-aware causal transformer introduced in Sec. [3.1](#S3.SS1) through three ablation studies, provided in Tab.[2](#S4.T2). First, we evaluate the impact of removing the proposed modules. Without the path-masking strategy, the model tends to overfit to path conditions, leading to degraded motion quality and weaker alignment with the text, as reflected by increased FID and reduced R-Precision. We also observe that introducing path masking improves generalization in path following. Further removing the proposed spatial state conditioning leads to an additional performance drop, confirming the importance of explicitly modeling spatial context during generation. Next, we examine the impact of the masking rate range $[r_{\min},r_{\max}]$ used during training. Our choice of $[0.5,0.9]$ outperforms both a conservative range $[0.1,0.5]$ and a broader range $[0.1,0.9]$, indicating higher masking rates more effectively promote balanced reliance on both path and text inputs. We analyze the effect of the contiguous segment length $\ell$ used for masking. Shorter segments, 5 points, lead to better path alignment and lower FID, whereas longer segments, 10 and 15 points, enhance text alignment at the cost of significantly degraded path-following accuracy. These results highlight a trade-off between semantic consistency and spatial control introduced by the masking strategy.

### 4.2 Performance of Generated 4D Scenes

Figure: Figure 5: Examples of generated 4D scenes. Texts above each scene are prompts for the environment (top) and motion (bottom) generation. See Supplementary Material for more examples and videos.
Refer to caption: /html/2508.12176/assets/x3.png

We evaluate the quality and diversity of the generated 4D scenes, which consist of indoor environments paired with human motion.
We generate 115 unique environments spanning diverse room types and objects, with 2 motion sequences synthesized per scene.
Objects are placed with a minimum clearance of 60 cm to ensure sufficient space for the human mesh to move inside.
In total, the generated environments include 920 unique objects, 25 objects per scene in average, 47 room categories, and 24 different dielectric materials. The average motion duration is 4.5 seconds.
We further provide qualitative visualization in Fig. [5](#S4.F5), showcasing diverse room layouts, furniture configurations, and semantically consistent human motion. These visualizations highlight the ability of our system to synthesize diverse 4D scenes in a scalable way.

### 4.3 Performance of Phase-Coherent Ray Tracing

To demonstrate the effectiveness of our phase-coherent ray tracing, we evaluate three benchmarks that require either spatial or temporal phase coherence.

Figure: Figure 6: Panoramic imaging results with and without spatial phase coherence.
Refer to caption: /html/2508.12176/assets/x4.png

Spatial Phase Coherence.
To benchmark spatial phase coherence, we consider a panoramic imaging setting that combines signals from 1,200 radar positions arranged evenly along a circular path.
Fig. [6](#S4.F6) shows the imaging results from two random scenes generated in the Sec. [4.2](#S4.SS2) with the beamforming algorithm in [24].
The improved image clarity highlights the role of spatial phase coherence, which ensures that wavefronts remain aligned across all radar positions. Notably, the ghost reflections in the imaging result show our simulation captures multipath effects that are difficult to produce in learning-based methods. These results demonstrate the importance of spatial phase coherence for downstream RF applications.

Figure: Figure 7: Recovered chest motion with and without temporal phase coherence.
Refer to caption: /html/2508.12176/assets/x5.png

Temporal Phase Coherence.
As mentioned in Sec. [3.2](#S3.SS2), stochastically casting rays results in complete failure in phase with motion. To enable a fair comparison, we cast rays fixed over time, which deviates from standard practice.
We first validate temporal coherence in a respiration tracking task.
We animate the chest of SMPL with varying poses following real breathing signals [25].
We generate in total 500-second signals using 40 breathing sequences.
As the chest moves, the minute distance change to radar can be detected by phase [59].
We extract the phase information in the simulated signal and convert it into distance change.
With temporal phase coherence, the reconstructed distance change curves achieve an RMSE of 0.08, and DTW of 8.89, compared with ground truth breathing data, significantly outperforming the baseline metrics, 0.14 and 12.68.
We also simulate a sphere undergoing sinusoidal motion and generate range–Doppler heatmaps using range and Doppler FFT. Our approach largely outperforms the baseline and we provide qualitative results in the Supplementary Material.
These results highlight the importance of coherent ray-surface mapping for stable phase tracking and reliable Doppler estimation in dynamic RF scenes.

### 4.4 Case Studies

To demonstrate the effectiveness of WaveVerse in the real world, we conduct two case studies on RF-based machine learning applications with publicly available data: high-resolution imaging [24] and human activity recognition [35].
Since collecting real-world RF data is costly and time-consuming, we evaluate each task under two conditions: a limited-data setting that reflects practical constraints, and a data-adequate setting where sufficient real-world samples are available.
Results show that the RF data generated by WaveVerse consistently improve performance in both scenarios.

High-Resolution Imaging.
We first evaluate a task that predicts dense depth maps from RF signals with ML [24].
It adopts a rotating radar as previously benchmarked in Sec. [4.3](#S4.SS3) on spatial phase coherence.
We adopt the same cross-building training protocol from [24], where the model is trained on RF data from 11 buildings and evaluated on 1,000 frames from a held-out building.
To enhance depth prediction in the limited-data setting, we augment the dataset with a larger scale of simulated RF signals and corresponding depth supervision generated by WaveVerse. We sample 1,000 real frames as a baseline dataset and progressively introduce $1\times$, $2\times$, and $4\times$ simulated data from 115 diverse environments synthesized in Sec. [4.2](#S4.SS2).

Figure: Figure 8: Performance comparison over the baseline with varying amounts of additional real and simulated data on: (a) high-resolution RF imaging and (b) human activity recognition. BL: Baseline.
Refer to caption: /html/2508.12176/assets/x6.png

Figure: Figure 9: Improved imaging quality over objects. (a) Armchair. (b) Water fountain. (c) Human.
Refer to caption: /html/2508.12176/assets/x7.png

Fig. [8](#S4.F8)(a) shows consistent improvements in MAE, 90th percentile error, and PSNR as the amount of simulated training data increases, outperforming the baseline trained with limited real data. With $4\times$ simulated data, MAE and 90th percentile error are reduced by 2.02 cm and 6.88 cm, respectively, while PSNR improves by 1.51 dB. These gains demonstrate that simulated data alone can effectively enhance performance in data-limited settings. For comparison, we include a $4\times$-real setting trained with 4,000 additional real samples. Notably, the simulated data captures 73.33% of the improvement in 90th percentile error achieved by the $4\times$-real model, and even surpasses it in PSNR.
Further analysis shows that our model excels in the high-quality ranges. 12.1% of our samples exceed $35\text{\,}\mathrm{dB}$, nearly twice the 6.6% achieved by the baseline. For a broader threshold of $30\text{\,}\mathrm{dB}$, the proportion of high-quality predictions increases from 41.9% to 45.4%. We visualize improvements in imaging quality over objects in Fig. [9](#S4.F9), which we attribute to the rich object diversity in our scenes.
Finally, combining simulated and real data yields the best performance, with an additional gain of 3.55 cm in 90th percentile error and $0.45\text{\,}\mathrm{dB}$ in PSNR, highlighting the value of WaveVerse-generated signals in both limited and sufficient data scenarios.

Human Activity Recognition.
We further evaluate WaveVerse on a downstream classification task that maps RF signal sequences to human activity [35]. To synthesize human motions, we leverage an LLM to generate diverse motion descriptions for five activities adopted in the dataset: walking, standing, squatting, jumping, and jumping jacks. We train a classifier on 100 real samples and evaluate on 500 held-out real samples, achieving a baseline accuracy of 31.6% shown in Fig. [8](#S4.F8)(b). As we progressively augment the training set with simulated data, adding 400, 900, and 1900 synthetic samples, the accuracy consistently improves to 49.8%, 61.4%, and 71.6%, respectively. This approaches the 75.6% accuracy achieved by training on the full set of 2,000 real samples. Finally, combining all available simulated and real samples yields the best performance of 81.0%.

## 5 Discussion

Limitation and Future Work.
While WaveVerse offers a scalable framework for simulating diverse RF signals in complex indoor environments, it has several limitations that present opportunities for future work. First, the current motion generation pipeline does not support tasks involving object interaction. A promising direction is to incorporate interaction-aware motion synthesis, potentially enabled by constructing large-scale datasets within our simulated environments. Second, the RF simulator currently omits complex wave phenomena such as diffraction and refraction, which could be integrated in future extensions to further improve the signal fidelity.

Conclusion.
We present WaveVerse, a prompt-based, scalable framework that generates dynamic 4D environments with human motion and simulates realistic RF signals via the proposed phase-coherent ray tracing.
Comprehensive evaluations and case studies demonstrate the practical utility of WaveVerse in enabling high-fidelity RF data generation and enhancing performance in both data-limited and data-adequate scenarios.
We will release our code and simulator to support future research in RF-based sensing and simulation.

Acknowledgments.
We thank Xin Yang, Haowen Lai, Zitong Lan, Yiduo Hao and Freddy Liu for their valuable comments and discussions that helped improve this work.

## References

- Achiam et al. [2023]
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*, 2023.
- Árbol and Casas [2024]
Baldomero R Árbol and Dan Casas.
Bodyshapegpt: Smpl body shape manipulation with llms.
*arXiv preprint arXiv:2410.03556*, 2024.
- Cai et al. [2020]
Hong Cai, Belal Korany, Chitra R Karanam, and Yasamin Mostofi.
Teaching rf to sense without rf training measurements.
*Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies*, 4(4):1–22, 2020.
- Cen et al. [2024]
Zhi Cen, Huaijin Pi, Sida Peng, Zehong Shen, Minghui Yang, Shuai Zhu, Hujun Bao, and Xiaowei Zhou.
Generating human motion in 3d scenes from text descriptions.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 1855–1866, 2024.
- Cendes [2016]
Zoltan Cendes.
The development of hfss.
In *2016 USNC-URSI Radio Science Meeting*, pages 39–40. IEEE, 2016.
- Chen et al. [2024]
Long Chen, Oleg Sinavski, Jan Hünermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton.
Driving with llms: Fusing object-level vector modality for explainable autonomous driving.
In *2024 IEEE International Conference on Robotics and Automation (ICRA)*, pages 14093–14100. IEEE, 2024.
- Chen et al. [2023]
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu.
Executing your commands via motion diffusion in latent space.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 18000–18010, 2023.
- Chen and Zhang [2023]
Xingyu Chen and Xinyu Zhang.
Rf genesis: Zero-shot generalization of mmwave sensing through simulation-based data synthesis and generative diffusion models.
In *Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems*, pages 28–42, 2023.
- Chi et al. [2024]
Guoxuan Chi, Zheng Yang, Chenshu Wu, Jingao Xu, Yuchong Gao, Yunhao Liu, and Tony Xiao Han.
Rf-diffusion: Radio signal generation via time-frequency diffusion.
In *Proceedings of the 30th Annual International Conference on Mobile Computing and Networking*, pages 77–92, 2024.
- Goodfellow et al. [2020]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.
Generative adversarial networks.
*Communications of the ACM*, 63(11):139–144, 2020.
- Guo et al. [2022a]
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng.
Generating diverse and natural 3d human motions from text.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 5152–5161, 2022a.
- Guo et al. [2022b]
Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng.
Tm2t: Stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts.
In *European Conference on Computer Vision*, pages 580–597. Springer, 2022b.
- Ha et al. [2020]
Unsoo Ha, Salah Assana, and Fadel Adib.
Contactless seismocardiography via deep learning radars.
In *Proceedings of the 26th annual international conference on mobile computing and networking*, pages 1–14, 2020.
- Ha et al. [2021]
Unsoo Ha, Sohrab Madani, and Fadel Adib.
Wistress: Contactless stress monitoring using wireless signals.
*Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies*, 5(3):1–37, 2021.
- He et al. [2025]
Hao He, Chao Li, Wolfgang Ganglberger, Kaileigh Gallagher, Rumen Hristov, Michail Ouroutzoglou, Haoqi Sun, Jimeng Sun, M Brandon Westover, and Dina Katabi.
What radio waves tell us about sleep!
*Sleep*, 48(1):zsae187, 2025.
- Ho et al. [2020]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
*Advances in neural information processing systems*, 33:6840–6851, 2020.
- Hoydis et al. [2023]
Jakob Hoydis, Fayçal Aït Aoudia, Sebastian Cammerer, Merlin Nimier-David, Nikolaus Binder, Guillermo Marcus, and Alexander Keller.
Sionna rt: Differentiable ray tracing for radio propagation modeling.
In *2023 IEEE Globecom Workshops (GC Wkshps)*, pages 317–321. IEEE, 2023.
- Hu et al. [2025]
Dongyin Hu, Xin Yang, Ahhyun Yuh, Zihao Wang, Lama A. Al-Aswad, Insup Lee, and Mingmin Zhao.
Tracking blink dynamics and mental states on glasses.
In *Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services (MobiSys)*, page 14, 2025.
- Hurst et al. [2024]
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al.
Gpt-4o system card.
*arXiv preprint arXiv:2410.21276*, 2024.
- Jakob et al. [2022]
Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang.
Mitsuba 3 renderer, 2022.
https://mitsuba-renderer.org.
- Jiang et al. [2023]
Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen.
Motiongpt: Human motion as a foreign language.
*Advances in Neural Information Processing Systems*, 36:20067–20079, 2023.
- Kaelbling et al. [1996]
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore.
Reinforcement learning: A survey.
*Journal of artificial intelligence research*, 4:237–285, 1996.
- Kingma et al. [2013]
Diederik P Kingma, Max Welling, et al.
Auto-encoding variational bayes, 2013.
- Lai et al. [2024]
Haowen Lai, Gaoxiang Luo, Yifei Liu, and Mingmin Zhao.
Enabling visual recognition at radio frequency.
In *Proceedings of the 30th Annual International Conference on Mobile Computing and Networking (MobiCom)*, pages 388–403, 2024.
- Li et al. [2024]
Yong-Xian Li, Jiong-Ling Huang, Xin-Yu Yao, Si-Qi Mu, Shou-Xin Zong, and Yan-Fei Shen.
A ballistocardiogram dataset with reference sensor signals in long-term natural sleep environments.
*Scientific Data*, 11(1):1091, 2024.
- Liang et al. [2023]
Kun Liang, Anfu Zhou, Zhan Zhang, Hao Zhou, Huadong Ma, and Chenshu Wu.
mmstress: Distilling human stress from daily activities via contact-less millimeter-wave sensing.
*Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies*, 7(3):1–36, 2023.
- Loper et al. [2023]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black.
Smpl: A skinned multi-person linear model.
In *Seminal Graphics Papers: Pushing the Boundaries, Volume 2*, pages 851–866. 2023.
- Loshchilov and Hutter [2017]
Ilya Loshchilov and Frank Hutter.
Decoupled weight decay regularization.
*arXiv preprint arXiv:1711.05101*, 2017.
- Nimier-David et al. [2019]
Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob.
Mitsuba 2: A retargetable forward and inverse renderer.
*ACM Transactions on Graphics (ToG)*, 38(6):1–17, 2019.
- Parker et al. [2010]
Steven G Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, et al.
Optix: a general purpose ray tracing engine.
*Acm transactions on graphics (tog)*, 29(4):1–13, 2010.
- Radford et al. [2018]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al.
Improving language understanding by generative pre-training.
2018.
- Radford et al. [2021]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In *International conference on machine learning*, pages 8748–8763. PmLR, 2021.
- Schulman et al. [2017]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.
Proximal policy optimization algorithms.
*arXiv preprint arXiv:1707.06347*, 2017.
- Shafir et al. [2023]
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano.
Human motion diffusion as a generative prior.
*arXiv preprint arXiv:2303.01418*, 2023.
- Singh et al. [2019]
Akash Deep Singh, Sandeep Singh Sandha, Luis Garcia, and Mani Srivastava.
Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar.
In *Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems*, pages 51–56, 2019.
- Sun et al. [2021a]
Yue Sun, Zhuoming Huang, Honggang Zhang, Zhi Cao, and Deqiang Xu.
3drimr: 3d reconstruction and imaging via mmwave radar based on deep learning.
In *2021 IEEE International Performance, Computing, and Communications Conference (IPCCC)*, pages 1–8. IEEE, 2021a.
- Sun et al. [2021b]
Yue Sun, Honggang Zhang, Zhuoming Huang, and Benyuan Liu.
Deeppoint: A deep learning model for 3d reconstruction in point clouds via mmwave radar.
*arXiv preprint arXiv:2109.09188*, 2021b.
- Sushil et al. [2024]
Madhumita Sushil, Travis Zack, Divneet Mandair, Zhiwei Zheng, Ahmed Wali, Yan-Ning Yu, Yuwei Quan, Dmytro Lituiev, and Atul J Butte.
A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports.
*Journal of the American Medical Informatics Association*, 31(10):2315–2327, 2024.
- Tevet et al. [2023]
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano.
Human motion diffusion model.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=SJ1kSyO2jwu](https://openreview.net/forum?id=SJ1kSyO2jwu).
- Van Den Oord et al. [2017]
Aaron Van Den Oord, Oriol Vinyals, et al.
Neural discrete representation learning.
*Advances in neural information processing systems*, 30, 2017.
- Vaswani et al. [2017]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
*Advances in neural information processing systems*, 30, 2017.
- Wan et al. [2024]
Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, and Lingjie Liu.
Tlcontrol: Trajectory and language control for human motion synthesis.
In *European Conference on Computer Vision*, pages 37–54. Springer, 2024.
- Wang et al. [2020]
Ge Wang, Chen Qian, Kaiyan Cui, Xiaofeng Shi, Han Ding, Wei Xi, Jizhong Zhao, and Jinsong Han.
A universal method to combat multipaths for rfid sensing.
In *IEEE INFOCOM 2020-IEEE Conference on Computer Communications*, pages 277–286. IEEE, 2020.
- Wang et al. [2022]
Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang.
Humanise: Language-conditioned human motion generation in 3d scenes.
*Advances in Neural Information Processing Systems*, 35:14959–14971, 2022.
- Wang et al. [2024]
Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang.
Move as you say interact as you can: Language-guided human motion generation with scene affordance.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 433–444, 2024.
- Watkins and Dayan [1992]
Christopher JCH Watkins and Peter Dayan.
Q-learning.
*Machine learning*, 8:279–292, 1992.
- Xie et al. [2023]
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang.
Omnicontrol: Control any joint at any time for human motion generation.
*arXiv preprint arXiv:2310.08580*, 2023.
- Xie et al. [2024]
Yiming Xie, Varun Jampani, Lei Zhong, Deqing Sun, and Huaizu Jiang.
Omnicontrol: Control any joint at any time for human motion generation.
In *The Twelfth International Conference on Learning Representations*, 2024.
- Xue et al. [2023]
Hongfei Xue, Qiming Cao, Chenglin Miao, Yan Ju, Haochen Hu, Aidong Zhang, and Lu Su.
Towards generalized mmwave-based human pose estimation through signal augmentation.
In *Proceedings of the 29th Annual International Conference on Mobile Computing and Networking*, pages 1–15, 2023.
- Yang et al. [2024]
Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al.
Holodeck: Language guided generation of 3d embodied ai environments.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 16227–16237, 2024.
- Yun and Iskander [2015]
Zhengqing Yun and Magdy F Iskander.
Ray tracing for radio propagation modeling: Principles and applications.
*IEEE access*, 3:1089–1100, 2015.
- Yun and Iskander [2024]
Zhengqing Yun and Magdy F Iskander.
Radio propagation modeling and simulation using ray tracing.
In *The advancing world of applied electromagnetics: In honor and appreciation of Magdy Fahmy Iskander*, pages 251–279. Springer, 2024.
- Zhang et al. [2023a]
Bin-Bin Zhang, Dongheng Zhang, Yadong Li, Yang Hu, and Yan Chen.
Unsupervised domain adaptation for rf-based gesture recognition.
*IEEE Internet of Things Journal*, 10(23):21026–21038, 2023a.
doi: 10.1109/JIOT.2023.3284496.
- Zhang et al. [2023b]
Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan.
Generating human motion from textual descriptions with discrete representations.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 14730–14740, 2023b.
- Zhang et al. [2023c]
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.
Adding conditional control to text-to-image diffusion models.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pages 3836–3847, 2023c.
- Zhang et al. [2023d]
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu.
Remodiffuse: Retrieval-augmented motion diffusion model.
In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 364–373, 2023d.
- Zhang et al. [2022]
Xiaotong Zhang, Zhenjiang Li, and Jin Zhang.
Synthesized millimeter-waves for human motion sensing.
In *Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems*, pages 377–390, 2022.
- Zhang et al. [2024]
Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, and Wanli Ouyang.
Motiongpt: Finetuned llms are general-purpose motion generators.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 38, pages 7368–7376, 2024.
- Zhao et al. [2016]
Mingmin Zhao, Fadel Adib, and Dina Katabi.
Emotion recognition using wireless signals.
In *Proceedings of the 22nd annual international conference on mobile computing and networking*, pages 95–108, 2016.
- Zhao et al. [2017]
Mingmin Zhao, Shichao Yue, Dina Katabi, Tommi S Jaakkola, and Matt T Bianchi.
Learning sleep stages from radio signals: A conditional adversarial architecture.
In *International conference on machine learning*, pages 4100–4109. PMLR, 2017.
- Zhao et al. [2021]
Mingmin Zhao, Kreshnik Hoti, Hao Wang, Aniruddh Raghu, and Dina Katabi.
Assessment of medication self-administration using artificial intelligence.
*Nature medicine*, 27(4):727–735, 2021.

## Appendix A Supplementary Material

In this supplementary material, we provide additional details and results of WaveVerse from three perspectives, conditional human motion generation, 4D world generation and RF simulation, following the structure presented in the main paper, along with the societal impacts.

Videos of all qualitative results can be found on the [project webpage](https://zhiwei-zzz.github.io/WaveVerse/).

### A.1 Conditional Human Motion Generation

Model Details.
Our model consists of two components, a VQ-VAE [40] tokenizer and the proposed state-aware causal transformer. The VQ-VAE is constructed using 1D convolutional layers, residual blocks, and ReLU activations in both the encoder and decoder along the temporal dimension. The temporal downsampling rate is set to 4, and the codebook size is $512\times 512$. The tokenizer is trained using the AdamW optimizer [28] with $\beta_{1}=0.9$ and $\beta_{2}=0.99$, for a total of 300K iterations and a batch size of 256. Following [54, 40], a combination of reconstruction loss, embedding loss, and commitment loss is employed. To further improve motion quality and training stability, velocity regularization along with exponential moving average updates and codebook resetting are adopted as described in [54]. The learning rate is initialized at 2e-4 and reduced by a factor of 0.05 after 200K iterations using a MultiStepLR scheduler.

The state-aware causal transformer is composed of 8 transformer layers [41], each configured with 8 attention heads and a hidden dimension of 512. To enforce temporal causality, we apply causal self-attention [31] throughout the network. Text conditions are encoded using CLIP [32], while path conditions and spatial states are encoded by a 3-layer MLP with a hidden dimension of 256. The model is trained to maximize the likelihood of token sequences using a cross-entropy loss, with a batch size set to 128. We optimize the transformer using the Adam optimizer with $\beta_{1}=0.5$ and $\beta_{2}=0.9$, and train for 300K iterations. The learning rate is initialized at 1e-4 and decayed by 0.05 after 150K iterations through a MultiStepLR scheduler.

Experiment Details.
We adopt HumanML3D [11] as our dataset, which contains 14,616 motion sequences annotated with 44,970 text descriptions. To extract path information, we downsample the pelvis trajectory into 64 evenly spaced 2D waypoints on the floor, which serve as the path condition. The dataset is split following the standard protocol as that in [39, 47, 54].
We adapt MDM [39], OmniControl [47], and T2M-GPT [54] as baseline models for our task. For MDM and OmniControl, we follow their original setup, where the target motion length is provided during both training and inference. To incorporate path information, we modify MDM by adding the encoded path features to its original conditioning inputs. For OmniControl, we replace the original per-frame joint features with shared path features. In both cases, we use the same 3-layer MLP as the path encoder to ensure a fair comparison. We experimented with both max pooling and mean pooling for aggregating path features, and found that max pooling yields better performance; thus, we apply max pooling when encoding paths for MDM and OmniControl. Additionally, for OmniControl, we implement a differentiable path extraction function to serve as its analytic function.
For T2M-GPT, we extend the input by appending path tokens after the text tokens, mirroring our own path condition encoding to ensure fairness.
For all other settings, including hyperparameters, we follow the original configurations.
All models are implemented in PyTorch and trained using an NVIDIA L40 GPU.

Training and Inference Time.
We report and compare our method with baselines for both the training and inference time.
We summarize the training time required in the Table [3](#A1.T3) below:

**Table 3: Comparison of training time (hours).**
| Ours | T2M-GPT | MDM | OmniControl |
| --- | --- | --- | --- |
| 27.1 | 19.7 | 20.9 | 47.1 |

In practice, we use the VQ-VAE checkpoint provided by [54] to ensure a fair comparison. As a result, the VQ-VAE training time, which takes 7.1 hours, is omitted from the table. While our method requires longer training time than T2M-GPT, due to the longer tokens, it is significantly more efficient than OmniControl, which achieves the closest performance.

**Table 4: Comparison of inference time (s). For autoregressive models, we report the time required to generate motion sequences of lengths 64 (first) and 196 (second).**
| Ours | T2M-GPT | MDM | OmniControl |
| --- | --- | --- | --- |
| (0.16, 0.47) | (0.09, 0.27) | 7.43 | 117.26 |

We report the inference time required to generate a sequence of human motion in Table [4](#A1.T4). Since the inference time for our method and T2M-GPT depends on the motion sequence length, we provide results for sequences of length 64 and 196. Experiments show that both autoregressive methods are significantly faster than diffusion-based approaches. While our method is slightly slower than T2M-GPT due to the need to compute spatial states on the fly, it delivers substantially better performance.

Qualitative Results.
We present qualitative results of our method for text- and path-conditioned human motion generation. We begin with customized conditions to highlight the capabilities of our model, followed by qualitative results from the test set of the HumanML3D dataset.

We begin by presenting qualitative results that demonstrate the model’s ability to follow diverse path lengths. To this end, we use the same text condition, walk, while varying paths of lengths [1, 3, 5, 7] meters, all oriented in the same direction. As shown in Fig. [10](#A1.F10), the input paths are visualized with colored points transitioning from blue to red to indicate temporal order. The generated motions closely follow the given paths while remaining consistent with the texts.

Figure: (a) Path length: 1m
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/paper-1m.png

We then change the text prompt to slowly walk and fix both the text and path length while varying the path direction by angles of $\pm 90^{\circ}$, $\pm 45^{\circ}$, and $\pm 30^{\circ}$. The corresponding visualizations are shown in Fig. [11](#A1.F11). The generated motions exhibit slower velocities compared to those in previous examples, reflecting the semantics of the updated text conditions. We refer the reviewer to the accompanying video for a clearer comparison. Additionally, the visualizations show that the generated motions accurately follow paths with varying directions, demonstrating strong path adherence.

Figure: (a) Path direction: $-90^{\circ}$
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/paper_-90.png

Next, we demonstrate the model’s ability to align motion with varying textual descriptions. Using a fixed path, we condition the model on different texts such as jump, run, walk as if there are stairs in the front, and wave their arms. As shown in Fig. [12](#A1.F12), the generated motions faithfully follow the same path while accurately reflecting the semantics of each instruction.

Figure: (a) Text: Jump.
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/paper-jump.png

Moreover, we showcase generalization to random combinations of different texts, path lengths and directions in Fig. [13](#A1.F13).

Figure: (a) Conditions: Run; 8m; $30^{\circ}$
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/paper-all-run.png

Finally, we present qualitative results of our model on the test set of the HumanML3D dataset, as shown in Fig. [14](#A1.F14). These ressults highlight the alignment between the generated motions and more complex text and path conditions, demonstrating the model’s ability to produce coherent and contextually accurate human motion. Generating these motions requires the model to jointly interpret the semantics of the text and the spatial constraints of the path, understanding not only what action is being described, but also where and how it should move.

Figure: (a) Text: The person takes a step and waves his right hand back and forth.
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/000650.png

### A.2 4D World Generation

Prompts.
We provide the adopted prompt in Fig. [17](#A1.F17) for the generation of motion descriptions and begin/end points. We also provide the prompts for the human shape generation in Fig. [18](#A1.F18) and the dielectric property generation in Fig. [19](#A1.F19), respectively.

Path planning. Given the input start and end points, we first generate a cost map from the scene layout, which is processed with morphological dilation. We then apply the A* algorithm to find a path between the start and end points. If no valid path is found, we regenerate the motion description along with new start and end points until a feasible path is obtained.

Qualitative Results.
While WaveVerse can effortlessly generate shorter motions in open or less constrained spaces, we emphasize its ability to handle more challenging scenarios, producing long, coherent motions within visually complex and spatially constrained environments. Figure [15](#A1.F15) showcases qualitative results in such cases, including narrow hallways, intricate layouts, and long human motions. The generated motions align with the surrounding layout, navigating obstacles and fitting within the scene’s geometry. Interestingly, the motions sometimes appear to interact with the scene, even though no explicit interaction is modeled.
Text prompts are provided in the subcaptions, and we refer to the project webpage for corresponding video visualizations.

Figure: (a) A broad gallery; Slowly tour around
Refer to caption: /html/2508.12176/assets/fig/supp_different_length/gallery.png

### A.3 RF Simulation

Simulator details.
The proposed phase-coherent ray tracing can be integrated into conventional ray tracing-based simulators. In our implementation, we build on Sionna [17], which uses Mitsuba 3 [20] as its underlying ray tracing engine. Sionna is released under the Apache 2.0 license, while Mitsuba 3 is distributed under a customized license. We adhere to the respective licensing terms of both libraries in our use and will ensure proper attribution and compliance when open-sourcing our customized phase-coherent ray tracing simulator.

Qualitative Results of Velocity Estimation from Doppler Effects
We provide qualitative results of velocity estimation from Doppler effects in the attached video (Doppler_comparison.mp4).

Figure: Figure 16: Temporally Coherent Ray Tracing.
Refer to caption: /html/2508.12176/assets/x8.png

In this experiment, we simulate a sphere moving back and forth along a straight line with sinusoidal velocity. A radar is placed in front of the sphere, and velocity is estimated based on Doppler shifts. The results are visualized as range-velocity maps at each timestamp. In these range-velocity maps, we expect to observe a sinusoidal velocity pattern over time, reflecting the periodic motion of the sphere.
Additionally, a narrow velocity band should span multiple range bins, as the sphere’s spatial extent causes different ranges to share the same velocity across its surface. The video clearly demonstrates that our method, which enforces temporal phase coherence, produces significantly cleaner range-velocity maps compared to conventional ray tracing approaches.

### A.4 Societal Impacts

We introduce WaveVerse, a scalable and physically grounded framework for simulating RF signals in dynamic environments. The ability to generate realistic RF data with diverse human motion and scene layouts has several potential positive societal impacts. It can facilitate progress in privacy-preserving sensing, indoor navigation and health monitoring by reducing reliance on vision-based sensors. By supporting high-fidelity simulation under varied conditions, WaveVerse may also help broaden access to RF research, lowering the barrier to entry for institutions without expensive hardware or large-scale data collection pipelines.
However, WaveVerse also raises negative societal impacts. Since WaveVerse relies on LLMs to generate human motions and semantic scene layouts, it inherits the risks associated with LLMs, such as biases in generated content, unintended reinforcement of stereotypes, which users should pay attention to.

Figure: Figure 17: Prompt for Motion Description and Path Generation.

Figure: Figure 18: Prompt for Human Shape Generation.

Figure: Figure 19: Prompt for Dielectric Property Generation.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. generalizability: In which circumstances would WAVEVERSE's performance degrade? How you intepret these cases?
soundness: 3
strengths: 1. Novel problem statement and formulation: RF signal in time series is really challenging.
2. Comprehensive dataset usage like Lai et al., 2024 for imaging and Singh et al., 2019 for activity recognition.
3. (Despite the overall work looks more like an itegration of existing techs rather than tacking theoretical challenges) The usage of LLM to align user's language with RF generation context and the implementation makes sense for potetial product use cases. The phase-coherence design aligns with SOTA.
4. Experiment setting makes sense to use generated data to boost current sensing task performance.
summary: This paper presents WAVEVERSE, a framework that generate realistic RF signals from pure text prompts. 
The system combines LLM-driven 3D scene and human motion generation with a phase-coherent ray tracing simulator that preserves spatial and temporal phase consistency, ensuring both physical fidelity and environmental diversity. 
Experiments show that generated data (when combined with or replacing limited real-world RF datasets) significantly improves model performance in both data-limited and data-adequate settings, demonstrating its value as a scalable source of high-fidelity synthetic RF data.
weaknesses: 1. Limited technical depth: The work primarily integrates existing components (LLM-based scene generation, motion modeling and ray tracing) rather than introducing new theoretical insights or core algorithmic innovations (like NeRF^2 (MobiCom'23)[1] and RF Genesis(SenSys'23))[2].

2. Lack of real-data calibration: The generation workflow is not calibrated or partially trained with real RF measurements, which could clearly have had the chance to help bridge the domain gap and validate simulation fidelity.

3. Unclear inference overhead. Although the algorithm looks not very heavy, it's still needed for a multi-modal model/workflow to demonstrate the computation overhead.

4. Lack of real-world case evaluation / case-specific analysis. Despite it may be too harsh to criticize the work for lacking real-world experiments, it is important to dive into concrete real-world examples (materials or layouts for which the workflow performs worse, any external factors like surrounding WiFi/cellular signal may interfere, etc) to identify the true potential of the work.

[1] Zhao, Xiaopeng, et al. "Nerf2: Neural radio-frequency radiance fields." Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.

[2] Chen, Xingyu, and Xinyu Zhang. "Rf genesis: Zero-shot generalization of mmwave sensing through simulation-based data synthesis and generative diffusion models." Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems. 2023.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: See Weaknesses
soundness: 3
strengths: Since RF Genesis, there have been few impressive papers in the field of RF simulation for quite some time. Overall, I hold a positive view of this paper for the following reasons:

- Case Study. I believe the most important aspect of evaluating RF simulation is the gain in real experiments. The current case study provides strong evidence of this. Based on this, I give a score of 6, and if the authors can address the weaknesses I have raised, I would be happy to increase it to 8 or higher.

- Introducing of Large Models. For RF simulation, introducing large models is an easily conceivable idea, but how to effectively introduce large models to improve system performance is not easy. The paper's approach to introducing large models is quite innovative and indeed effective.

- Signal Simulator. The newly designed signal generator demonstrates a certain degree of innovation.
summary: This paper addresses the critical challenge of acquiring large-scale, high-quality datasets for RF sensing by introducing WAVEVERSE, a scalable, prompt-based framework for simulating realistic RF signals in dynamic 4D worlds. The system integrates two main innovations: a language-guided 4D world generator and a physics-based, phase-coherent signal simulator. Experiments validate this approach, showing the phase-coherent simulation yields high-fidelity signals for beamforming and respiration monitoring. Case studies on high-resolution RF imaging and human activity recognition demonstrate that WAVEVERSE-generated data not only enables RF imaging simulation for the first time but also consistently improves downstream task performance in both data-limited and data-adequate settings.
weaknesses: 1. My primary concern focuses on the RF baseline methods: As mentioned earlier, I believe the case study is very important. However, the current case study only demonstrates that the proposed method is effective compared to having no simulation data. But is it effective compared to data generated by other simulation methods? I suggest adding comparative experiments with data generated by other simulation methods.

2. The paper should provide more evidence that the motions generated by the LLM are physically plausible in terms of dynamics; it only proves that the motions are semantically aligned. A motion that "looks like slipping" does not mean it is physically possible. Similarly, the dielectric properties assigned by the LLM based on semantics have not been physically validated. The "realism" foundation of this framework (LLM input) is semantics-based, while its simulator (ray tracing) is physics-based; the paper does not validate the "physical realism" gap between these two domains.

3. There are some potentially overclaimed aspects in the paper:

(1) How to define "data-adequate." The paper claims to improve performance in data-adequate scenarios. Theoretically, when data is sufficient, the improvement from simulation data should be minimal. If the improvement is significant, then the data is not sufficient.

(2) "Data generation for RF imaging for the first time" is also strange. Although data generation for RF imaging is difficult, it is not necessarily the first time."

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: 1. Do you calibrate generative geometry/materials to match measured RF responses? Any domain-gap mitigation (e.g., distribution alignment, correction nets)?
2. Sensitivity to prompt wording, motion model, mesh resolution, material catalog size; which factors dominate RF error?
3. How do you model material EM properties, antenna patterns, phase noise/CFO, timing offsets, and human-body scattering?
soundness: 2
strengths: 1. If the world/motion generator is language-guided, it could make scenario coverage and data diversity dramatically easier.
2. Potential to unify CV-style 4D generative assets with RF rendering, bridging two active communities.
summary: This paper proposes a framework for scalable RF simulation inside generative 4D (space–time) worlds, featuring a language-guided world generator and a state-aware causal transformer for human motion (“WaveVerse”). It is novel to couple generative scene/motion synthesis with RF propagation to accelerate scenario creation at scale.
weaknesses: 1. No verified evidence here of RF accuracy vs. ground truth (ray tracing/EM solvers/measurements).
2. Treatment of multipath, diffraction, penetration, materials, etc, unclear.
3. Generators may induce distribution shift; need calibration showing RF outputs remain physically plausible under varied prompts.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please see the points raised in the Weaknesses section.
soundness: 3
strengths: + The integration of LLM-driven environment and motion generation with a phase-coherent ray-tracing simulator is thoughtfully designed and demonstrates strong engineering effort, providing a scalable pipeline for synthesizing diverse RF datasets.

+ The work is conceptually innovative in its vision of bridging generative AI and physical simulation, laying a solid foundation for future research that combines semantic scene synthesis with RF propagation modeling.
summary: This paper presents WAVEVERSE, a scalable framework that generates realistic RF sensing data by combining LLM-driven 4D world generation (text-based scene and motion synthesis) with phase-coherent ray tracing for physically accurate signal simulation, enabling diverse and high-fidelity RF datasets synthesis.
weaknesses: - Although the framework is well engineered, its core innovations build on existing foundations such as LLM-based 3D environment generation, SMPL-driven motion synthesis, and conventional ray-tracing techniques. The work would be stronger with clearer methodological advances that are unique to RF simulation, for example through learned physical priors, adaptive ray selection, or a formal analysis of phase coherence that distinguishes it from prior generative or physics-based systems.

- The idea is interesting and potentially impactful, yet the method inherently faces a fundamental limitation: it remains unclear how closely the simulated RF data align with real-world RF measurements. Because the 4D scene generation process creates entirely synthetic environments and motions that cannot be exactly replicated in the physical world, there is no definitive way to quantify the realism gap between simulated and real signals. Although the downstream case studies show encouraging performance gains, these results do not fully establish physical correspondence, leaving uncertainty about how faithfully WAVEVERSE captures real propagation characteristics.


- The path-based motion conditioning, while scalable, lacks explicit temporal control, which may result in unrealistic motion timing or velocity patterns that undermine physical plausibility in dynamic scenes. Moreover, the phase-coherent ray tracing, though conceptually well-motivated, lacks formal error analysis to verify its phase preservation accuracy.


Decision: Reject