FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens
Authors: Yiming Zhong
Yumeng Liu
Chuyang Xiao
Zemin Yang
Youzhuo Wang
Yufei Zhu
Ye Shi
Yujing Sun
Xinge Zhu
Yuexin Ma
Venue: NeurIPS 2025 poster ()
Keywords: Robot Action Generation
Visuomotor Policy Learning
Frequency-domain Modeling
Continuous Latent Representation
Primary area: deep_learning
TL;DR: This paper explores the representation of robot actions in the frequency domain and introduces FreqPolicy, a novel frequency-domain autoregressive framework  for hierarchical robotic action generation.

Abstract: Learning effective visuomotor policies for robotic manipulation is challenging, as it requires generating precise actions while maintaining computational efficiency. Existing methods remain unsatisfactory due to inherent limitations in the essential action representation and the basic network architectures. We observe that representing actions in the frequency domain captures the structured nature of motion more effectively: low-frequency components reflect global movement patterns, while high-frequency components encode fine local details. Additionally, robotic manipulation tasks of varying complexity demand different levels of modeling precision across these frequency bands. Motivated by this, we propose a novel paradigm for visuomotor policy learning that progressively models hierarchical frequency components. To further enhance precision, we introduce continuous latent representations that maintain smoothness and continuity in the action space. Extensive experiments across diverse 2D and 3D robotic manipulation benchmarks demonstrate that our approach outperforms existing methods in both accuracy and efficiency, showcasing the potential of a frequency-domain autoregressive framework with continuous tokens for generalized robotic manipulation.Code is available at https://github.com/4DVLab/Freqpolicy

FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens
Yiming Zhong', Yumeng Liu?, Chuyang Xiao', Zemin Yang', Youzhuo Wang'. Yufei Zhu', Ye Shi², Yujing Sun Xinge Zhu Yuexin Ma
SShanghaiTech University "The University of Hong Kong
8 Digital Trust Centre, Nanyang Technological University"The Chinese University of Hong Kong
fzhongym2024, mayuuexin wew.hhhhhiihhehcccceecc edu.
Project Page: httpgss//feeeggoollc..eetttboo.
Code: https:
Abstract
Learning effective visuomotor policies for robotic manipulation is challenging, as it requires generating precise actions while maintaining computational efficiency. Existing methods remain unsatisfactory due to inherent limitations in the essential action representation and the basic network architectures. We observe that repre- senting actions the frequency domain captures the structured nature motion effectively: low-frequency components reflect global movement patterns, while high-frequency components encode fine local details. Additionally, robotic manipulation tasks of varying complexity demand different levels of modeling precision across these frequency bands. Motivated by this, we propose a novel paradigm for visuomotor policy learning that progressively models hierarchical frequency components. To further enhance precision, we introduce continuous latent representations that maintain smoothness and continuity in the action space. Extensive experiments across diverse 2D and 3D robotic manipulation benchmarks demonstrate that our approach outperforms existing methods in both accuracy and efficiency. showcasing the potential of a frequency-done domann autoregressive framework with continuous tokens for generalized robotic manipulation.
Introduction
The study of visuomotor policies enable robots to learn task execution from demonstrations by leveraging raw visual inputs, such as images or point clouds, allowing them to generate effective action sequences in response new visual observations. It has become prevailing paradigm in robot manipulation [10). However the requirement of high precision in robotic tasks and the sequential correlation in action space present challenges for visuomotor policy learning.
Existing methods for visuomotor policy learning can be broadly categorized into diffusion-bain based methods [10, 48 38
Diffusion- based approaches model the action distribution conditioned on observations, leveraging their powerful generative capabilities to produce reliable and diverse action sequences even from limited demon- strations. However, diffusion models typically encounter higher computational costs and increased inference latency due to their iterative sampling process, which could limit their practicality in cfficiency-sensitive applications. In contrast, AR methods sequentially predict each action step conditioned previous actions and current observations. They generally more computation- ally efficient and enable faster inference, making them attractive for real-time control scenarios. Nevertheless, AR approaches may be prone compounding errors over long horizons, and due to their common reliance on discrete representations, often struggle to accurately model inherently continuous action spaces, limiting their ability to capture complex temporal correlation. Despite
42
47
[5]
43 39 and autoregressive (AR) methods [33 16 49
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

recent advances, both diffusion-basse and AR methods share a fundamental limitation: they overlook the diversity of the action space arising from task complexity and the degrees of freedom in robotic manipulation. Representing the inherent structured features of different actions is of great significance for the generalization and robustness of visuomotor policies.
To address these limitations, we rethink the robotic action represen- tation from a frequency-domain perspective. Figure I shows one example of the action signals (from Adroit Door (30) reconstructed
Orighia Signal
Froveccy (8-00%)) Renan
Keargy: 9.9% RADNE-A22
by filtering different frequency bands. We can see that the first 30%
of frequency bands is already sufficient to allows us to reconstruct
Mii Frepaccy (8-387)) Recemisaticn
Energn K.M RABSE--11
signal nearly identical to the original, preserving 95% of its energy, Retaining just the first 10% of bands can recover the general trend of the signal. Incorporating high-frequency information, up to the first 60% of frequency bands, enables the restoration of finer details, such as subtle oscillations. Meanwhile, through extensive statistical analysis of robot task execution behaviors, we found the required frequency components vary depending on the complexity of differ- ent tasks. For simple tasks, such as pick-and-place, fow-frequency information is often sufficient, and fegghfrreunncy signals may be redundant, especially the collection of demonstrations in real world can introduce noise or unnatural jitter. Filtering out these components results in smoother, more natural actions. However, for complex tasks such as dexterous manipulation, figh-frequency details are essential for precise control. In summary, for continuous action spaces, investigating hierarchical frequency-dome domain information and developing effective representation and modeling methods for spectral features are crucial for enabling robots to perform tasks with varying complexity levels.
High Fropacicy -90i Renembtuiice
Energg 8.%% RoesE-aae
eayy (1-10096)) Recomstrction
Energy
Figure 1: Action signals re- constructed across different fre- quency bands; see Appendix for energy details.
With this observation, we propose to learn the visuomotor policies based on the modeling of hierar- chical frequency domains. Notably, low-frequency information is easier to learn and captures the global structure of the motion. Rather than producing the entire frequency spectrum at once, we adopt a multi-stage progressive approach, starting from low- wwwffrequency representations and gradually extending to full-spectrum actions. This corse-to- trssett-fine generation process not only simplifies the modeling complexity of sophisticated actions but also provides wwwffrquency actions with protection from high-frequency noise, ensuring stability in the fundamental motion structure. Autoregressive (AR) paradigm, with their sequential modeling capabilities, are inherently well-suited to implement this coarse-to-fine generation paradigm, effectively capturing hierarchical dependencies between frequency domains. However, previous AR-based methods [29] usually discretize the inherently continuous action space, resulting in significant information loss. Recent study [19] also suggests that discretization of originally continuous spaces is not necessary for AR modeling, and diffusion models can better representing continuous probability distributions. Motivated by these insights, we propose a coarse-to- fine robotic action generation paradigm, FregPolicy, that combines continuous action representations with AR modeling in the frequency domain.
To be specific, our approach utilizes the Discrete Cosine Transform (DCT) 18 to convert action sequences into frequency components. Leveraging a masked encoder-decoder architecture, we map trajectories from various frequency bands into distinct latent codes. It predicts refined motions progressively, where low-frequency signals guide the generation of high-frrquency details. Our framework bridges the gap between hierarchical frequency representations and probabilistic modeling, offering unified and scalable solution for visuomotor policy learning. Extensive experiments show that our method yields significant improvements on challenging robotic manipulation benchmarks, demonstrating both its efficiency and state-of-the- performance for generating precise and high- fidelity actions. In summary, our contributions are as follows:
We have explored frequency- -bsed action space representation and propose a novel solution for effective visuomotor policy learning by progressively modeling hierarchical frequency components to capture the structured nature of robotic motions.
We introduce continuous latent representations with diffusion-based decoding, eliminating discretization requirements while preserving action space continuity and autoregressive efficiency.
Our method achieves ttate-of-the-art performance on extensive robotic benchmarks, signifi- cantly outperforming others in both success rate and computational efficiency.

---

2 Related Work
2.1 Action Representations for robots
Action representation is important in robotic learning, as it determines how agents encode and generate behaviors in complex environments. Recently, transformers have achieved remarkable success in large language models (8 [[6],, demonstrating exceptional versatility in sequence generation tasks. At the same time, diffusion models have shown strong generative capabilities for images 17 44 and have garnered significant attention for their adaptation to robotic policies [TO, 48). The advancements in transformer and diffusion models have given rise two primary paradigms for action representation: discrete and continuous. Transformer models typically employ discrete tokenization, encoding actions as sequences of tokens. In robotic tasks, discretization can be realized through straightforward quantization of each action dimension at every timestep 77 6 or clustering
approaches such as BeT (32, which enable the generation of diverse behaviors but may sacrifice fine-grained control.
In contrast, diffusion models and other continuous approaches 33
50
[4] leverage
probabilistic Srahe as VAEs, diffusion processes, and normalizing flows-to to model the action space directly. These methods preserve the full expressiveness and precision of the original action space, avoiding the loss of nuance that can result from discretization. However, this advantage often comes at the cost of increased computational complexity, particularly in nigh-dimensional settings.
Recently, frequency-domain strategies have been explored address redundancy in both images 28 20 455 and action spaces. FAST [29] demonstrates that excessive high-frequency information in action sequences can hinder model training, and propose compression-based tokenization scheme that reduce redundancy in action signals and improves training efficiency. However, both frequency- domain compression and action discretization inevitably cause information loss, which limit fine- grained, precise synthesis. In this work, we combine frequency-domain action representation with continuous sequence modeling. Our approach enables flexible transitions between spatial and frequency domains, preserving critical details for high-fidelit motion while maintaining efficiency, thus supporting more expressive and scalable robotic policy learning.
2.2 Visuomotor Policy for Robotic Manipulation
Visuomotor policies for robotic manipulation map visual inputs directly to control actions, enabling robots to interact with their environments in closed feedback loop. Broadly, there are main approaches to tackling this problem: one is to use diffusion based methods, and the other is to employ autoregressive methods. Diffusion-based methods typically generate action segments by modeling the conditional distribution of actions given observations using diffusion models [10 48, 38, 42).
Autoregressive methods generate action sequences step-by-step, predicting each action based on previous outputs [3] 40). While this approach is efficient, it often lacks long-horizon structural modeling. Recently, several coarse-to-fin methods have been proposed, but they primarily operate in the temporal domain. CARP [6 incorporates multi-scale reconstruction from VAR (35) for action generation, but depends on discrete representations derived from VQ-VAE 37.. which limits precision. DensePolicy [44] introduces bidirectional, BERT-inspired [11 framework for hierarchical, coarse- to-fine action prediction. However, its iterative refinement relies on progressively increasing temporal upsampling density, which does not remove in action signals. Unlike previous methods, our approach performs AR generation in the frequency domain, modeling signals at different frequencies independently. This separation prevents low- frequency components from being affected by high-frequency noise. Moreover, for high-dimensional and complex tasks, progressively learning high-frequency signals from fww-ureuy components eases the challenge of direct generation.
noise compress redundant information
3 Method
3.1 Overview
Problem Formulation Given a dataset of paired sequences D = {(o, x)). where each o .N_... denotes sequence of observations (c.g., RGB images, depth maps, or point clouds)
3

---

20/22/333 Observatie Ensoder
Sequences
At
X0
X.
k-level sdiin
Frog/elicy Encoder
FregPolicy Decoder
Condition 2
per token
8-level Coeffiiient
for P(r)s)
(c) FreePolicy Training
Il, Novel
Iter
Indes: Beg mollawss
(leer sulpat) Full-freuamcyy
Full-freqazney
Nobe Xr
10
1y*
Sagoo
y/v
Encoder
Decoderr
Cood
Toleenior
Gei Geleel
DCT IDCT
Expo Mo
Robot Stane
steps
Oxiput Acrion
Inde--1
L...
Palicy Sumpling
Indox
(b) FreePolicy Inference.
Alovel recorstruction action token
Full-Boquency actios token
Indox tokeo
Observation tokcn
Masiedd tokes
Condition sokes
Figure 2: Pipeline of Freqcolicy showing both training (a) and inference (b) procedures. We first transform action trajectories into the frequency domain via DCT, and then learns latent codes for different frequency level actions using FreqPolicy, and reconstructs actions through masked prediction and a diffusion-based decoder. This enables robust, frequency-aware, and high-fidelity robotic action generation.
E IR Nxd represents the corresponding sequence of robot actions with variable length N and action dimension d. our objective is train a policy that produce action trajectories & based on new observation sequences ó to effectively perform the demonstrated tasks.
and each x = [Zo, NN -I
T1....
As shown in Figure 2 our method consists of two stages. During training (Figure 21), we apply DCT to decompose action sequences into frequency components and reconstruct them at different levels via inverse DCT. For frequency level k. the FreqPolicy encoder-decoder processes the k-level reconstruction y", observation features, and frequency index produce continuous token z which conditions a diffusion model to reconstruct the original full-frequency action x. We adopt frequency-ave masking (gray blocks in the figure) to improve training efficiency. During inference (Figure 2)) we perform hierarchical generation through Niter iterations. Starting from zero (y = 0). each iteration generates full-frequency action & conditioned on the previous output y then filters to y for the next iteration. This coarse-to-fine process progressively refines actions from low-frequency global structure to high-frequency fine details.
This section is organized as follows: Section 3.2 analyzes action trajectories the frequency domain: Section 3.3 introduces the space-frequency transformation method employed in approach; Section 3.4 describes the continuous token representation with diffusion models; and Section 3.5 presents the frequency- based hierarchical generation strategy.
3.2 Frequency Domain Analysis
To motivate our approach, we first conduct a systematic analysis of action trajectories under various task conditions. We selected two representative benchmarks to evaluate our approach. For complex manipulation tasks, we chose three tasks from the Adroit Benchmark [30], which utilizes dexter- ous robotic hand with 26 degrees of freedom. For simpler tasks, we selected three tasks from the Robomimic Benchmark [23), which features parallel grippers with only degrees of freedom. these tasks, we aim to analyze the energy distribution of each action dimension of the robotic arm across different frequency bands, as well as to investigate the impact of discarding high-frequene signals task performance.

---

HighhDimenssonal Space 220) Adrit
Adve Elanner
14
Adice Pro
Adie Dooe
Edkegg 8o000
03
9-0%
13008
0404
14-10%
30-64
10
36485
56-12%
0.
06
Low- Low-Dimensional Action Space (D 18)
Can MH
Square Mf
Lih MH
Actiun Dimension
Stecds Balt
0.4
Con MiI Squre
0185
10.3%
36.5%%
18-00%
6.0%
10-00%
30.00%
30.00%
600%
30.30%
33-30%
50-00N
8.2 8.90
Frequency Bund
Froquency Band
Frequency Band
Frequency Domais Scale
(a)
(b)
Figure 3: (a) Heat maps of frequency band energy across action dimensions for different tasks. The top row shows Adroit 30 tasks with high-dimensional actions (26 dimensions), while the bottom
presents Robomimic (23 tasks with low-dimensional actions (10 dimensions). (b) Success rate
of actions reconstructed with varying frequency ratios. We reconstruct action sequences using different proportions of frequency components and evaluate their success rates the original tasks.
Energy Distribution. We present the energy heat maps for these tasks in Figure 3 (a), where the vertical axis represents different dimensions of the action space, corresponding to the joint angles of the robotic hand arm, and the horizontal axis indicates the energy ratio across different frequency bands. The red regions indicate areas with high energy ratio, suggesting that these frequency bands contain greater proportion of the signal's energy. Conversely, the blue regions represent areas with low energy ratio. In both the Adroit tasks (top row) and the Robomimic tasks (bottom row), most of the energy is concentrated in the 0-10% freeuueunncy bands (leftmost columns), which is why these regions appear red. By comparing the heat map for tasks in different action space, we found that tasks with dibedniiiiionll actions show greater variance in the energy distribution compared to oow-dimensional tasks. Specifically, the energy distribution of the action space in the Adroit Benchmark shows distinet low-frequency (0- (0%10% patterns across its three tasks, while in the Robomimic tasks, the low- oowfrequency (0-10%) energy distribution remains relatively uniform with little variation among the three tasks.
Performance under Frequency Compression. To further assess the role of high-frequene, infor- mation on task performance across different benchmarks, we compress action signals by removing the figh-freeuency components from the original sequences, reconstructed the actions with varying proportions of low-frequency information, and measured their success rates on the original tasks. The results, shown in Figure ())) indicate that omitting high-freeg feeh-ffrruencyy signals leads significant drop in success rates for Adroit tasks. In contrast, -ww-dimeniional tasks in Robomimic maintain stable performance with as little as 12% of low-frequency information preserved. This analysis underscores the importance of frequency-awart modeling: the necessity of high-frequency signals for effective action representation varies across different scenarios.
Conclusion. These findings provide the empirical foundation for our frequency-based autoregressive generation strategy. They suggest that adjusting the granularity of frequency-domain representations can help optimize model performance for tasks with different levels of complexity. Our goal is to enable the model learn individual task skills from expert demonstrations in diverse action spaces and with varying task complexities. Therefore, we first transform the actions into the frequency domain and recover them at different frequency scales. This allows the model predict subsequent actions using information from lower frequency bands, enabling coarse-to- to-fine generation process.
3.3 DCT Decomposition
Our spectral analysis reveals that the proportion of frequency components required for effective action representation varies across different tasks. Therefore, it is necessary to transform the action space from the time domain the frequency domain, for more effective robotic action modeling. To efficiently achieve this, employ the Discrete Cosine Transform (DCT), which can project
5

---

time-domain trajectories onto set of cosine basis functions with low computational cost. This provides a compact and interpretable frequency-domain representation, enabling hierarchical mod- eling of actions in various tasks settings and scenarios, Specifically, given an action sequence x [oo 21; ................................ x E TR-xd, where each In E Rd represents the action at time step n, we transform it into the frequency domain by applying the DCT independently to each action dimension. For each dimension, the DCT is defined as:
N-1
1 2 2) i 1= N
X; = Σ
In cos
π n+
(I)
Here, X; denotes the i-th DCT coefficient. Given any integer i such that 1 S k < N, we refer to retaining only the first & DCT coefficients as preserving the k-level DCT. With these & coefficients, the original trajectory can be approximately reconstructed using the inverse DCT, shown in Equation 2
yn
1 N
Xo +
Σ
k-1
X cos
(n+22) (#(
(n+
n n 0, O.., N-1
Specifically, let y = [ud ....... - I Nxd denote the k-level reconstruction of the original sequence x. If l: = N, meaning all DCT coefficients used, the reconstruction will recover the original trajectory without any loss of information. However, if k < N, only the lowest discarded. As result, the
frequency components are kept, and all higher-frequecc components reconstructed trajectory y becomes a compressed and smoother version of the original, preserving the essential structure of the original sequence while discarding minor fluctuations.
3.4 Continuous Tokens for FreqPolicy
We explore integrating FreqPolicy with continuous token representations using a diffussin model. Compared to discrete tokens, continuous tokens offer greater expressiveness and enable more fine- grained modeling of the action space, allowing for smoother and more accurate representation of complex trajectories. Accurately modeling the probability distribution of each token is essential for effectively incorporating a continuous tokenizer into autoregressive models. Building on MAR [19], tackle this challenge by adopting diffusiin-based loss function for training and devising specialized frequency-aware sampling mechanism for efficient inference. Additional details on our training and inference processes are provided in the appendix.
Training During training, given an action sequence x E R. of length N, we first apply the Discrete Cosine Transform (DCT) to obtain its frequency coefficients, as defined in Equation [ and compute reconstructions at all frequency levels as described in Equation 2 Our objective is to train a model capable of modeling token distributions across different frequency levels. For an arbitrary frequency level k, we encode the k-level reconstruction y", the observation o, and the level index k using the FreqPolicy encoder to obtain latent representation. This latent feature is then decoded by the FreqPolicy decoder to produce a condition vector zf, which serves as continuous token for y We subsequently model the conditional probability p(x x²*) using a reverse diffusion process, which allows for conditional generation based on zt recover the original action sequence. This recovered sequence can then be further processed with DCT and inverse DCT to obtain reconstructions at any desired frequency level.
Sampling The sampling procedure follows the standard inference process of diffusion models. Beginning with initial noise sampled from a standard normal distribution, the diffusion tokenizer iteratively removes the noise to produce & ~ p(x z²). Here, & denotes trajectory sampled from the action space of demonstration data with complete frequency components. This enables flexible reconstructions at various frequency levels, by simply specifying the desired frequency level k. Diffusion Loss The diffusion tokenizer optimized using the diffusion loss introduced by [19] as presented in Equation 4
x(=*,x)= = E,,, [| [|c--o(xx (x 1, 1,=
where x, is a noise-perturber version the original trajectory x, e a noise vector sampled from (O I), and Eg is small MLP parameterized by A that predicts the added noise. Gradients are propagated through 2², enabling end-to-end optimization of FreqPolicy encoder, decoder and the noise predictor.
Masked Generative Strategy To reduce training cost and enhance generation diversity, adopt

---

the frequency-awer aware masking mechanism from FAR 45]. Since early autoregressive steps process information-speae sparse, low-frequency inputs, using all tokens is redundant. The frequency-awan mask- ing strategy in FAR 45] applies higher mask ratios at lower-frequency levels, gradually increasing the number of tokens as higher- frequency information is incorporated.
3.5 Frequency-based Hierarchical Generation
Our previous analysis demonstrated that compressing actions in the frequency domain at different scales reveals hierarchical information. Actions reconstructed with higher compression ratio are smoother and better capture the overall trends of the motion, which helps reduce noise interference and lowers the difficulty of generation. Based on this observation, we propose a frequency- based autoregressive strategy for hierarchical action generation, as illustrated in Figure 2 increasing frequency level sequence {4o, .,.... .N........ J where (0 = 0 (starting from zero input) and IN N (full frequency). Each I represents the number DCT coefficients retained at iteration i. Starting from lo, we iterate from low to high frequencies Nuer times to generate the action sequence. In each iteration i E {1, corresponding -level reconstruction action tokens y are encoded to produce continuous token - that represents the current context. The diffusion model then samples a full-frequency action sequence x conditioned on this token. For the next step, we use the 4--leel reconstruction of 8', i.e., y", the input for the following iteration, which contains richer frequency information than the previous input.
We define an
2, ..., N. J. the observation tokens, the frequency level index 4-1, and the
Example. For 16-timestep trajectory with Nuer = 4 iterations, we might use {40 = 0,4 4, 8,13 = 12.4, 16). At iteration we start with y" (all zeros) and generate x!, then filter keep only the first 4 DCT coefficients, obtaining y" for iteration 2. This process continues, progressively adding more frequency components (4 + 8 >> 12 > 16) until the final full-frequency output.
This hierarchical generation paradigm first captures the basic trends and global structure of the action sequence using low-frequency components, and then progressively enriches the output by incorporat- ing higher-frequency signals that model fine-grained motion details. By explicitly decomposing and modeling motions at different frequency levels, our approach allows the model to better learn and represent both global dynamics and subtle variations in complex action sequences. This structured frequency decomposition, combined with autoregressive conditioning, enables more efficient and expressive generation of diverse and realistic motions.
4 Experiments
This section provides comprehensive evaluation of our proposed method. We first describe the experimental setup, including benchmarks, baseline methods, and implementation details. Next, we analyze the frequency domain requirements of different tasks and highlight their unique characteristics. We then compare our method in both the time and frequency domains. Additionally, we benchmark our approach against autoregressive methods with continuous and discrete token representations across various simulation benchmarks, and present results from real-world applications. Finally, we discuss the inference speed and ablation in sampling.
4.1 Experimental Setup
Benchmarks. We evaluate our methods on a diverse set of benchmarks that provide different types of observation data. Benchmarks with only 2D image observations are referred to as 2D tasks, which include two single-task benchmarks, Robomimic (33 and Push T [12]. Benchmarks with 3D visual observations are referred to as 3D tasks, consisting of Adroit [30]. Dex [4], Meta World [46], and RobTwin (26). which together cover wide range of robotic manipulation and dual-arm collaborative tasks. Tasks in DexArt are conducted using Allegro Hand 3 with 22 DoF, tasks in Adroit use ShadowHand [3I] with 26-28 DoF, and tasks in RoboTwin using dual-an arm grippers with DoF. Other tasks use parallel grippers with 10 DoF.
Baseline. For 2D tasks, we compare against Diffusion Policy (DP) as well as two autoregressive approaches using discrete token representation: Behavior Transformer (BeT) [32] and CARP [16]. DP available in two variants: CNN-based (DP-C) and Transformer- (DP-T). For 3D tasks, we use 3D Diffusion Policy (DP3) [48 and Mamba Policy [9] as baselines.
7

---

Implementation Details. Our model can be seamlessly integrated into the codebases of Diffusion Policy (DP) and 3D Diffusion Policy (DP3). To ensure fair comparisons, we use the same parameters and observation input processing as Diffusion Policy for the 2D tasks, and as 3D Diffusion Policy for the 3D tasks, maintaining consistency with their respective frameworks. Our approach enables flexible adjustment of autoregressive iteration counts in the frequency domain during inference. For all simulation experiments, we use iterations, whereas for real-world experiments, only I iteration used.
Table 1: Success rate (%) comparison on 10 simulation tasks in Adroit, DexArt and Meta- World. To ensure fair comparison, all tasks employ full-spectrum outputs and identical frequency level progression. denotes results we reproduced using the same expert demonstrations as ours.
Adroit
30
DexArt
Meta-Work
[6
Task
Hammer
Door
Pen
Laptop
Faucet
Toilet
Bucket
Assembly
Disassemble
Stick Push
Average
Diffusion Policy
45=5
37=2
13+2
69c4
23+s
58+2
46=1
15=1
13=7
63=a
44.0
DP3
100+0
02+4
83-1
6343
82s4
46-2
97=4
74.4
Mamba Policy
100.0
6841
41
802
3322
76g
27
100
76.4
100
70.1
DP3
1001e
5312
5015
83.3
33.12
70.6
24.14
95.a
87.3
$3.3
67.8
Mamba Policy+
100:0
59te
55:2
79:3
35:6
65
23:22
96:2
90:2
82:9
68.4
ours(ur/o DCT)
100.e
5115
4719
55.6
23112
451.
3013
95.0
80.3
61.4
ours
100:0
65:5
59:5
85:4
301s
77:55
25:24
97:2
92:8
85:
71.5
Table Main results on 48 simulation tasks. The table reports average success rates (%) across all tasks. denotes results we reproduced using the same expert demonstrations for fair comparison. Success rates for individual tasks are in Appendix.
Adroit (3)
DexArt (4)
MetaWorld Easy (20)
MetaWorld Medium (11)
MetaWorld Hard (5)
MetaWorld Very Hard (5)
Average
Algorithm \
Task
DP3
68.3
68.5
91.7
61.6
38.0
49.0
62.9+16.9
Diffusion Policy
31.7
49.0
86.8
31.1
10.8
26.6
39.3+23.9
DP3#
67.7
52.5
89.9
66.8
42.8
68.0
64.64114.7
Mambu P
71.3
50.5
90.2
67.4
46.4
69.0
65.8+14.4
ours
74.7
54.3
92.4
67.4
48.8
70.2
67.9=14.2
Table 3: Success rate ( (%) comparison on the RoboTwin Benchmark for Dual-Arm Manipulation with D435 Camera Setting. We evaluated our approach on 7 tasks using 20 expert demonstrations and seeds (0, 1, 2) and report the success rate. Our method was compared against DP3 (XYZ+RGB) and DP. all tested under the same conditions. Both DP and our method are trained for 500 epochs, while DP3 is trained for 3,000 epochs.
Rohu/Twin 20
Block Hammer
Block Handover
Bottle Adjust
Container
Empty Cup Place
Pick Messy
Dual Bottles Pick(Hand)
B.lano
6.3aso
B.lann
25
$7.3
27.7
7:0
4.2 Simulation Result
High Success Rate, In Table T we compare our method on 10 tasks from Adroit, DexArt, and MetaWorld, while Table 3 presents results 7 RoboTwin tasks using colored point clouds as observations. In Table [ our method outperforms both the state-of-the-arr (SOTA) methods and a baseline (without DCT decomposition)
8 tasks, and achieves comparable results the remaining 2. Similarly, as shown in Table 3 our method achieves superior performance in RoboTwin, with tasks reaching higher success rates than DP and DP3. We also evaluated all 48 tasks in Adroit, DexArt, and MetaWorld and reported the average success rate for each benchmark. As shown in Table 2 our approach maintains an average success rate of 67.9 99,, surpassing Diffusion Policy 3D (64.6%) and Mamba Policy (65.8%). These results demonstrate the effectiveness of our approach across variety of task settings and scenarios. More detailed results for each individual task are provided in the appendix.
Generalization Ability, Results Table 4 demonstrate that our method achieves consistently strong performance in unseen DexArt test environments, maintaining high success rates under both limited (10 demonstrations) and abundant (100 demonstrations) data settings. These results demonstrate the method's robustness and generalization ability in diverse scenarios.
8

---

Table 4: Generalization Results. Success rates (%) on unseen DexArt test data after 3000 training epochs, averaged over 100 trials x 3 random seeds.
10 Demonstrations
100 Demonstrations
Alg \
Task
Laptop
Faucet
Toilet
Bucket
Laptop
Faucet
Toilet
Bucket
DP3 Mamba Policy
12.2
11.523
8.5
15.0+3
36.4=5
17.4
25.4
29. 21
11.9=3
8,7
11.2+4
23.6+4
30.9=5
16.6#4
26.8=s
25.9+3
ours
14.5=3
12.4:4
16.6:4
32.3
40#3
12.1+3
27.8
26.4+1
Table 5: Noise Robustness Evaluation. Success rates (%) with relative change vs. high-quality baseline (see Appendix Table TO shown as superseripts. We evaluate two noise conditions: (1) Low- Quality demonstrations using suboptimal trajectories (5 tasks, hammer excluded), and (2) Gaussian
Noise (std=0.025, 0.05, 0) applied to high-quality baseline demonstrations (6 tasks).
Method
Low Low-Quality Demonstrations
Gaussian Noise std=0.025
Pick
Soccer
Stick
Hammer
Door
Avg.
Pick
Stick
Hammer
Door
Pen
Avg.
DP3
10-17
15-28
41-20
47
39-31
29-40
26+
25*14
63
0-200
65th
58
37-21
Mamba
16-36
20
40 27
21-23
34-38
31-41
18-24
33
61+11
0-200
50-15
57
37-12
Ours
20
32 45
82
32
00-8
70
52
56
11
Method
Gaussiun Noise std-0.05
Ganssiun Nuise std-0.1
Soccer
Stick
Hammer
Door
Avg.
Pick
Stick
Hammer
Door
Pen
Avg.
DP3
21
19-14
51-11
0-160
27-49
50
29-41
7-12
18-1s
50-12
0-1m
24-35
88-21
25-1a
Mamba
23
25-11
60
0-100
51
57
36-11
3-88
14-30
39-29
0-300
21-44
39-20
19-44
Ours
25
30
60-3
51
18
46
55"
45
20
27 16
52
15
32
49 16
Noise Robustness. We evaluate robustness under two noise scenarios (Table 5:: (1) training on low-quality demonstrations with reduced reward thresholds (5 tasks used, hammer task cannot generate low-reward expert demonstrations), and (2) adding Gaussian noise (std=0.025, 0.05, 0.1) to high-quality demonstrations. On low-quality demonstrations, all methods show significant performance decline, validating the importance of high-qualit training data. More importantly, under Gaussian noise conditions, our method demonstrates exceptional robustness: high noise (std=0.1), we achieve 33% success versus 19-23% for baselines. The Hammer task is particularly revealing- baselines completely fail (0% success) under any Gaussian noise, while we maintain 70%, 51%, and 15% at increasing noise levels. This superior performance stems from our low-frequency to high-frequency progressive generation mechanism's 's natural anti-noise advantages: (1) Low-frequency anti-noise characteristics-lov low-frequencc components inherently have stronger noise resistance, establishing a stable action foundation in initial stages; (2) Progressive filtering-each frequency domain level's reconstruction process acts as natural noise filtering: (3) Structured constraints the coarse-to- fine generation process ensures that key action structures are preserved even under noise interference. Interestingly, moderate noise (std=0.025) can promote generalization in certain tasks: for Pen spinning and Pick Out of Hole, low-intensity Gaussian noise unintentionally increases hand movement variation, adding action diversity and leading to higher success rates than noise-free baselines. This suggests beneficial noise augmentation can enrich the action distribution.
Table 6: Success rates ) comparisons on Robomimic benchmark (4 and Push-T task 12]
Tokanzzr
Mode
Visaa
Littzn)
Scaee
Viual
Cancrhii
Staee
Vicaal
Squarsiest
Stee
Viail
Stato
Visaal
Paa-T
State
Tixe
Mudea
DP-C
1.001.000
1.000.97
1,0000 96
1,000.96
0.98%0884
0.970082
0.890869
0.68/0.46
0.910.84
0.950091
2.11 0.95/0.9
De-1
1.000.99
1.001.000
1,000098
1.00194
0.9400800
0.950081
0.7300.00
0.62/0.35
0.780066
Anterggretre Madiie
1.000199
1,000.0
0AR80.45
0.21/0066
Dazdd
DARP
0.940.900
1.000.97
0.74/b.6
0.88A885
0.460.42
0.4400577
0.00WD000
0.06/0.00
0.880089
0.850183
0.09
Dentreuee
Dars:
1.00/1.00
1.000.98
0.9813.44
1.000.00
0.8480.88
0.880074
0.580050 50
0.5000.38
0.820776
0.928.85
0.21
Discrete vs Continuous. As shown in Table 6 our systematic comparison between discrete autoregressive and continuous diffusion models reveals a key finding: discrete representation methods (CARP, BET) exhibit significant limitations in modeling continuous action spaces, primarily due to information loss during the discretization process, particularly evident in hard tasks requiring precise control. In contrast, our proposed approach that integrates autoregressive and diffusion mechanisms in continuous representation framework not only maintains consistent high performance across various tasks, but also inherits the computational efficiency advantages of autoregressive methods (requiring only 1/10 of DP's inference time), conclusively demonstrating its superior expressiveness,

---

computational efficiency. and environmental adaptability in modeling continuous action spaces. Inference Efficiency. As shown in Table 6 our continuous token AR method with integrated diffusion mechanisms achieves an order of magnitude faster inference (0.21s vs. 2.1 compared to diffusion policy while maintaining comparable success rates on the Rohomimic benchmark.
Flexible Sampling. Our method allows flexible selection of the number of autoregressive iterations in the frequency domain during inference. We evaluated the impact different iteration counts on task success rates and inference times the Adroit benchmark, and provide Pareto frontier visualization [22] in Figure 4 which displays the tradeoff between inference time and success rate, meth-
76
Betoer
DP3
74-
Suacer Buanc
Dero
ods closer to the top-left corner achieve better trade-offs between efficiency and performance. It shows that method maintaining competitive performance even at minimal iterations.
00.14 07.35
Der
Dor
04
4.3
Real-World Result
Iefirenc Thm O4
Figure Pareto Analysis on Adroit benchmark. The x- axis represents inference time and the y-axis indicates task success rate.
To validate our method in real-world upplications, we collected demonstration data for an object handover task using the teleopera- tion system of the ShadowHand [31. In this setup, human subject holds an object, and the robot required to stably receive it. The dynamic interactions between the hand and object necessitate high inference speed for real-time responsiveness. Our real-world tests, conducted on an RTX 4090 GPU, show that single-iteration implementation achieves 70 FPS, significantly outperforming DP3's 25 FPS. As illustrated Figure 5 our robotic hand successfully receives the object from the human subject. These results demonstrate the practical advantages of our method for real-time robotic control applications with strict temporal constraints. Additional experimental details for real-world scenarios are provided in the appendix.
Figure 5: Real-Worid Experiments on Robotic Handover The robotic hand stably receives an object from human subject during real-world testing.
5 Conclusion
In this paper, we introduced FreqPolicy, a novel visuomotor policy framework leveraging hierarchical frequency-doman representations and continuous tokens for effective robotic manipulation. By decomposing action signals into frequency components using DCT, our method allows for flexible reconstruction of actions under different levels of action details. By integrating continuous latent representations with an autoregressive paradigm, our method enables precise and efficient modeling of action spaces via coarse-to- fo-fiee generation, eliminating discretization losses. Extensive experiments demonstrate that our approach achieves state-of of-the-art performance, outperforming existing methods in both success rate and computational efficiency. In future work, plan to extend research to the VLA framework, further exploring how frequency domain representations influence action spaces in multi-task training environments.
6 Limitations
Since all of our experiments used condition inputs consistent with DP3 DP, we have not yet explored the potential impact of altering condition input methods model performance. Additionally, it should be noted that our method still has room for improvement 2D tasks, and performance tends to decrease when frequency domain partitioning becomes too fine-grained.
10

---

7 acknowledgements
This work was supported by NSFC (No.62206173), Shanghai Frontiers Science Center of Human- centered Artificial Intelligence (ShangHAD,, and MoE Key Laboratory of Intelligent Perception and Human-Machine Collaboration (KLIP-HuMaCo).
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08874, 2023.
[2] AgileXRobotics. Agilex robotics. btttp:://global.ggtt ai/ 2025. Accessed: 2025-05-
16.
[3] AllegroHand. Allegro hand robot hand. http:://ww..lleerohaad com/ 2024. Accessed: 2025-05-16.
[4] Chen Bao. Helin Xu, Yuzhe Qin, and Xiaolong Wang. Dexart: Benchmarking generalizable dexterous manipulation with articulated objects. In CVPR, 2023a.
[5] NVIDIA: Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang. Dieter Fox, Fengyuan Hu. Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jianman Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. GrOOt nl: An open foundation model for generalist humanoid robots, 2025.
[6] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, al. Rt-2: Vision-language- action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15811 2023.
[7] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06881 2022.
[8] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[9] Jiahang Cao, Qiang Zhang, Jingkai Sun, Jiaxu Wang, Hao Cheng, Yulin Li, Jun Ma, Yecheng Shao, Wen Zhao, Gang Han, et al. Mamba policy: Towards efficient 3d diffusion policy with hybrid selective state models. arXiv preprint 2xX1:22409007633, 2024.
[10] Cheng Chi, Siyuan Feng. Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. RSS, 2023.
[11] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pages 4171-41888 2019.
[12] Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning. Conference on Robot Learning (CoRL), November 2021.
[13] Letian Fu, Huang Huang, Gaurav Datta, Lawrence Yunliang Chen, William Chung-Ho Panitch, Fangchen Liu, Hui Li, and Ken Goldberg. In-context imitation learning via next-token prediction. arXiv preprint arXiv:2408. 15980, 2024.

---

[14] Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In arXiv, 2024.
[15] Niklas Funk, Julen Urain, Joao Carvalho, Vignesh Prasad, Georgia Chalvatzaki, and Jan Peters. Actionflow: Equivariant, accurate, and efficient policies with spatially symmetric flow matching. arXiv preprint urXiv:2409.04570 2024.
[16] Zhefei Gong, Pengxiang Ding, Shangke Lyu, Siteng Huang, Mingyang Sun, Wei Zhao, Zhaoxin Fan, and Donglin Wang. Carp: Visuomotor policy learning via coarse-to-fine autoregressive prediction. arXiv preprint arXiv:2412.06782, 2024.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
[18] Syed Ali Khayam. The discrete cosine transform (dct): theory and application. Michigan State University, 114(1):31, 2003.
[19] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He.. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37:56424-56445, 2024.
[20] Xiaoyi Liu and Hao Tang. Difffno: Diffusion fourier neural operator. arXiv preprint arXiv:2411.09911, 2024.
[21] Yumeng Liu, Yaxun Yang. Youzhuo Wang, Xiaofei Wu, Jiamin Wang. Yichen Yao, Sören Schwertfeger, Sibei Yang, Wenping Wang. Jingyi Yu, Xuming He, and Yuexin Ma. Realdex: towards human-like grasping for robotic dexterous hand. In Proceedings of the Thirty- -hird International Joint Conference Artificial Intelligence, IJCA '24, 2024.
[22] Alexander V Lotov and Kaisa Miettinen. Visualizing the pareto frontier. In Multiohjective optimization: interactive and evolutionary approaches, pages 213-243. Springer, 2008.
[23] Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martin. What matters in learning from offline human demonstrations for robot manipulation. arXiv, 2021.
[24] Ajay Mandlekar, Danfei Xu, Josiah Wong. Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martin-Martin. What matters in learning from offline human demonstrations for robot manipulation. In Conference on Robot Learning (CoRI.), 2021.
[25] Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martnn. What matters in learning from offline human demonstrations for robot manipulation, 2021.
[26] Yao Mu, Tianxing Chen, Zanxin Chen, Shijia Peng, Zhiqian Lan, Zeyu Gao, Zhixuan Liang, Qiaojun Yu, Yude Zou, Mingkun Xu, et al. Robotwin: Dual-arm robot benchmark with generative digital twins. arXiv preprint arXiv:2504. 13059, 2025.
[27] Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots, 2024.
[28] Mang Ning, Mingxiao Li, Jianlin Su, Haozhe Jia, Lanmiao Liu, Martin Beneš, Albert Ali Salah, and Itir Onal Ertugrul. Detdiff: Intriguing properties of image generative modeling the det space. arXiv preprint 2rXiv:2412.15033, 2024.
[29] Karl Pertsch, Kyle Stachowicz, Brian Ichter. Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747. 2025.
[30] Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv, 2017.
12

---

[31] ShadowHand. Shadow dexterous hand series research and development tool. https://ww/ celdoorrooo..com//eetteeeeeee haassshand--aaa serddss 2024. Accessed: 2025-05-16.
[32] Nur Muhammad Shafiullah, Zichen Cui, Ariuntuya Anty Altanzaya, and Lerrel Pinto. Behavior transformers: Cloning & modes with one stone. Advances in neural information processing systems, 35:22955-22968, 2022.
[33] Avi Singh, Huihan Liu, Gaoyue Zhou, Albert Yu, Nicholas Rhinehart, and Sergey Levine. Parrot: Data-driven behavioral priors for reinforcement learning. arXiv preprint 4rXiv:2011.10024, 2020.
[34] Yue Su, Xinyu Zhan, Hongjie Fang, Han Xue, Hao-Shu Fang. Yong-Lu Li, Cewu Lu, and Lixin Yang. Dense policy: Bidirectional autoregressive learning of actions. arXiv preprint arXiv:2503.13217 2025.
[35] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-84865 2024.
[36] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 1.XXi::30021397// 2023.
[37] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning, Advances in neural information processing systems, 30, 2017.
[38] Chenxi Wang, Hongjie Fang, Hao-Shu Fang, and Cewu Lu. Rise: 3d perception makes real- world robot imitation simple and effective. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 287022808200 2877. IEEE, 2024.
[39] Dian Wang, Stephen Hart, David Surovik, Tarik Kelestemur, Haojie Huang. Haibo Zhao, Mark Yeatman, Jiuguang Wang, Robin Walters, and Robert Platt. Equivariant diffusion policy. arXiv preprint arXiv:2407.01811 2024.
[40] Yuqing Wang, Zhijie Lin, Yao Teng, Yuanzhi Zhu, Shuhuai Ren, Jiashi Feng, and Xihui Liu. Bridging continuous and discrete tokens for autoregressive visual generation. arXiv preprint arXiv:2503. 16430, 2025.
[41] Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies expressive policy class for offline reinforcement learning. ICLR, 2023.
[42] Shijie Wu. Yihang Zhu, Yunao Huang, Kaizhen Zhu, Jiayuan Gu, Jingyi Yu, Ye Shi, and Jingya Wang. Afforddp: Generalizable diffusion policy with transferable affordance. arXiv preprint arXiv:2412.03112. 2024.
[43] Jingyun Yang, Zi-ang Cao, Congyue Deng, Rika Antonova, Shuran Song, and Jeannette Bohg. Equibot: Sim equivvarart diffusion policy for generalizable and data efficient learning. arXiv preprint arXiv:2407.0m 01479, 2024.
[44] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming- Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. arxiv 2022, arXiv preprint 2XXi:2209900096 2022.
[45] Hu Yu, Hao Luo, Hangjie Yuan, Yu Rong, and Feng Zhao. Frequency autoregressive image generation with continuous tokens. arXiv preprint arXiv:2503.05305, 2025.
[46] Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-worid: A benchmark and evaluation for multi-task and meta reinforcement learning. In CoRI., 2020.
[47] Yanjie Ze, Zixuan Chen, Wenhao Wang. Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, and Jiajun Wu. Generalizable humanoid manipulation with improved 3d diffusion policies. arXiv preprint 2xXiv:2410.10803 2024.
13

---

[48] Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. In Proceedings of Robotics: Science and Systems (RSS), 2024.
[49] Xinyu Zhang, Yuhan Liu, Haonan Chang, Liam Schramm, and Abdeslam Boularias. Autore- gressive action sequence learning for robotic manipulation. IEEE Robotics and Automation Letters, 2025.
[50] Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Garrett, and Ajay Mandlekar, Spire: Synergistic planning, imitation, and reinforcement learning for long-horizon manipulation. arXiv preprint arXiv:2410.18065 2024.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]. .
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The abstract and introduction accurately describe the proposed framework
FreqPoley, contributions. and experimental scope which align with the content presented in the paper|section section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification:
We will discuss the limitations of our work in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We elaborate our methodology in detail in section as well as disclose our implementing details in ssection 4
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Currently, the code is not publicly available, but we will release it along with detailed instructions upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.ccppulic//guut http://icc/guiiee///eeeuubiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The implementation details (data splits, hyperparameters, optimizers, etc.) of all the simulation experiments in this paper remain exactly the same as in previous work (presented in subsection 4.Th, while the specific details of the real-world experiments described the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The experimental part of this paper is consistent with previous work, i.e., we report the error bars of our experiments where they report the error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
18

---

We will report the computational resources required for all experiments in the Appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We confirm that the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our method does not pose any negative societal impacts. The potential positive societal impacts have been discussed in the |section 1} our approach can advance the development of vision-driven strategies for robotics, thereby facilitating progress in real-world robotic applications.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
19

---

11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: For all the priors works quote and compare, we make reference to their original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pawerssipppweiiiie...... thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
20

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is important, original, or non-standard component of the core methods in this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The core method does not involve L.MSs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy http://errppp.cc//oooeeeeeee///p for what should or should be described.
21

---

Appendix
This document contains supplementary materials for our main paper. We provide further technical details, additional experimental results, and more qualitative examples to complement the findings presented in the main text. We hope this supplementary information will help readers better understand our approach and results.
The remainder of this supplementary material is organized as follows. In Section we provide the hardware specifications used in our experiments. In Section [ we list the hyperparameters employed. Section
BC presents the detailed algorithms for training and inference. In Section D we describe the ablation studies conducted. Section D outlines the details of our real-world experiments. Finally, in Section F
we offer further discussion on VLA models and frequency domain analysis.
A
Computational Resources
To ensure reproducibility, we provide detailed information on the computational resources used in experiments. For all simulation environment experiments including training, inference, and time benchmarking tests, we used NVIDIA RTX 2080Ti GPUs. Our model has 63M parameters, with DP3 255M, consuming approximately 4.5GB of memory during operation real-world environment experiments, we employed NVIDIA RTX 4090 GPUs for training, inference, and time benchmarking tests.
B Hyperparameters
In T we present the hyperparameters used in our experiments. For the baseline methods DP and DP3, we use their default hyperparameters. For the Adroit, DexArt, and MetaWorke benchmarks, our models are trained for 3,000 epochs. For the Robomimic and Push-T tasks, we use 1,000 training epochs, and for the RoboTwin benchmark, our models are trained for 500 epochs.
Table Hyperparameters used for various benchmark.
Hyperparameter
Value
Horizon (T))
16 for RoboTTwn)
Action step (Ta)
8 (6 for RoboTwin)
Observation step (T,)
(3 for RoboTwin)
point_ feature dim
64
state Sate_mlp_size
Batchsize
128
Num_iter
Sifps(Dino training)
100
Num_ gaon_ssmllingggtee steppfiisnt sampling)
ddim10
Diffloss_d
3
Diffloss_w
1024
encoder_embes eeweeemmeedddeee dim
512
decoder_embed dim
512
perodderddppth
decoder_depth
encoder_num_ heads
Optimizer
AdamW W
Betas (31, 32))
[0.95, 0.999]
Learning Rate
1.0e-4
Weight Decay
1.0e-6
Learning Rate Scheduler
Cosine
22

---

C Training and Inference Details
The training process for FreqPolicy is outlined in Algorithm T At each epoch, first encode the input observations using an observation encoder. The ground truth action sequence is then transformed into the frequency domain via the Discrete Cosine Transform (DCT), and a frequency index is randomly sampled. Conditional reconstruction is performed by applying the inverse DCT up to the sampled frequency level, enabling the model to focus different frequency components during training. An adaptive mask ratio is determined based on the frequency index, and a mask sampled accordingly. The masked observation and conditional reconstruction are then encoded and subsequently decoded by the FreqPolicy encoder and decoder, respectively. The diffusion model trained predict noise added to the actions at randomly sampled diffusion steps, using a standard mean squared error loss. Model parameters are updated by minimizing this loss throughout the training epochs. This procedure effectively leverages masked autoregressive modeling and diffusion-based generation, enabling the policy to learn robust representations across the frequency domain.
Algorithm FreqPolicy Training
Require: Number of training epochs K, Observation Encoder Eobs. FreqPolicy Encoder E, FreqPol- icy Decoder DD diffusion model eg, Observation O, Ground truth action x, Horizon T, diffusion steps Taiss, initial mask ratio m.
1: for == to K do
2: Zobe f (O)
>> Encode observations > Apply DCT, sample index
3: (Xo. XI...., XT-1) + DCT(X), k ~ U(0, T)
4: we [IDCT((Xo X1, XI...., xx-1)) if > 0 if k 0
Extract k-level reconstruction
5:
mask ratio m kKTk/T)
Adaptive mask ratio
6: mask ~ TruncNorm(mask -_tiioratooo
7:
Zmask + E y² k; mask)
D. Encode with mask Decode with mask
8:
zk D((obss Zmask, k; mask)
9:
~ ((1,
(0,
10:
X₁ + V v1 11-10 ācc
11:
C E., -e - ea 60(x,, 1, )+
b Diffusion loss
12: Update model parameters by minimizing L
13:
end for
return Trained models Eobs, E, D. and EB
The inference process for FreqPolicy detailed in Algorithm 2
Given an input observation, we first encode it using the observation encoder. The action tokens are initialized to zeros and are fully masked the beginning. For each iteration, the model progressively predicts actions at increasing frequency levels, guided by the current frequency index.
At each step k, the partially reconstructed tokens and the current mask passed through the encoder and decoder to obtain the continuous latent code z² The diffusion sampler then generates an updated action prediction conditioned on z*_ If it is not the final step, the predicted actions transformed into the frequency domain using the Discrete Cosine Transform (DCT), and the reconstruction is refined up to the next frequency level via inverse DCT. The masking ratio is adaptively reduced at each iteration, gradually revealing more of the reconstructed action sequence. This process continues until the entire action sequence is fully reconstructed. By progressively incorporating information across different frequency bands, our inference procedure enables the policy to generate high-fidelity predictions.
D Ablation
To verify the effectiveness of our proposed method and the contribution of each component, we conducted series of ablation experiments.
The results in Table present the ablation study on the prediction horizon, it shows that the perfor- mance is not sensitive to this parameter, demonstrating the robustness of our method. When Tj, 8
23

---

Algorithm 2 FreqPolicy Inference
Require:
Observation Encoder Eoha. FreqPolicy Encoder E, FreqPolicy Decoder D, DiffusionSam- pler F, diffusion steps Taiss. Observation O, Horizon T, Number of iterations Nster Frequency
indices {do, "...i....,iN.. ..r
Initialize:
2: mask + 1BxT
> Full masking initially
3: tokens <- OBXT DBXTTxDactoon
Zero initialization
4: Zobs t- Eobss (O))(O)
> Encode observations
5:
for step to Niter do
6:
k f- isep
Current frequency index
7:
Zmask t {Zobaa tokens, k, mask)
8:
z¹ (Zober Zmaski k, mask)
9:
& x D-=((2 kk,z,,k,7 Taiss)
Generate prediction via diffusion
10:
if step < Niter then
{Xo, ,I....,X7 .T..., + DCT(X)
Transform to frequency domain b Next frequency level
12:
next_ 4- istep411
tokens IDCT((X)p XI...., xexx
14: else
tokens X
16:
end if
17:
mask ratio t- cos
step+1
2
Niter
18:
mask + GenerateMask carissmmmmsaatttaa
end for
20:
return tokens
>> Final action sequence
or Th = 16, the model achieves slightly better performance with average over 58 points, while both smaller and larger horizons yield comparable results. This robustness across a range of temporal scales highlights the flexibility of our approach. Notably, only at extremely long horizons (T) = 64) do we observe noticeable decline in performance.
Table presents the results of the ablation study masking strategies. Our frequency policy mask significantly improves performance across all tasks, increasing the average score from 40 to 58-an improvement of approximately 45%. These results clearly demonstrate the importance of the frequency-based masking strategy for prediction tasks.
Table 8: Ablation study on prediction horizon. Analysis of Horizon (Tk), Action step (T.), and Observation step (T,).
Tb
To
To
Hammer
Door
Pen
Pick Out of Hole
Soccer
Stick Pull
Average
2
6546
76+3
5545
37+4
23+4
60+2
53+17
4
2
2
62+4
71+4
5346
31
38#3
64#2
53+14
8
2
4
100+0
68+2
52+4
25+2
38+4
6240
58+24
8
2
6
100+0
72+4
50
29+3
31
6443
58+25
16
2
10040
6545
59+5
30+2
3244
6240
58428
16
2
12
100+:
59+4
5143
3542
35::4
55:5
56:221
32
2
16
98+2
58+5
38+2
3443
1942
52+4
50+25
64
2
32
80+4
35+4
42+2
35+5
32#3
38+4
44+17
Table 9: Ablation study on mask. This experiment analyzes the model performance with and
without mask.
Hammer
Door
Pen
Pick Out of Hole
Soccer
Stick Pull
Average
W/o mask
99
3514
3113
1510 30
2712
3412 6240
40:27
Freqpolicy
100
65-1
5945
32#4
58+23

---

Table 10: Main results on 48 simulation tasks. Success rates (%) for each task are provided in this table.
(Eary)
Coffee
Open
Dierr Unlock
Open
DP3
100-a
100.a
Mi
100.u
Dillusion Policy
100.0
100.a
DP10
100lsa
10ltao
1005s
58an
1000.0
47as
10030
100l:
100l.n
100ls0
Marbh Policy*
100ke
100tan
100
50m4
100z.n
Missi
100se
1001
100m
100ten
100ca
100
100
72
100
71
100ee
100g
100
100en
(Bacr)
Alg's
Open
Hundle Prass
Plau
Stile
Sidk:
Reash
Window
Window Open
DDP
100zz
100tee
Tilx
100e11
Wien
100kcn
100ke
M4
100la0
100:
Ditrasion Policy
100.0
4015
8534
100.o
100.0
18.5
100.0
100..
Dess
100kn
Miss
Maa
100ee
100cn
100een
100lee
27
100lee
100
Marnba Policy"
100:0
T4le
100.e
100 .a
100.0
100.c
172.
100.0
100..
oEES
100
90sa
$4.4
100.e
100.0
100
100.e
30.2
100.e
100.o
Meta- World (Medium)
Alg Tisk
Hammer
Peg Insert Side
Push Wall
Scccer
Sweep
Sweep
Busketball
Bin Picking
Close
Cottee Pull
Coffee
Dips
49a8
1545
Diffusion Palicy
1549
34s7
852a
81a5
62s1
87a8
22:3
100:0
Mamta Policy"
68
0266
1548
100.0
26-33
48ain
80aa
ours
96
97
32s4
8524
10:3
83
31
56
100
01as
Meta World (Hard) Pick Out Hole
World (Very Hard)
Assembly
Hand
Place
Shalf Place
Disaesemble
Push
Place
09:1
14a9
12s4
17.10
09h04
27as
35as
Diffission Policy
18:13
9a2
Oze
Osu
30ts
11as
43s7
11a2
63s5
Sex
LMPS
9643
1142
12s7
42
b4s4
21a4
87
078a
8363
92
Mamba Policy+
9628
18:8
25a8
36a10
60g4
38.4
90
55:9
82ss
80
oues
9758
1753
80
57
63
2750
9230
0250
8546
8has
Adroit 30
DexArt 4
Average (41+3++4)
Alg \ Task
Hammer
Door
Pen
Laptop
Faucet
Toilet
Bucket
DP3
100
62#4
4326
8341
63-2
82.14
4642
71.36
Diffusion Policy
45a5
3742
13:2
6944
2348
58
46:1
53.25
DP3*
100
53+2
50
8343
3342
70
24
72.91
Mamba Policy*
100=0
59+3
55#2
7943
3546
65ss
23+2
73.71
ours
100=0
65+5
59+5
85+4
3043
7743
25+3
75.54
E Real-world Experiment Details
In real-world experiments, we employ an RGB-D camera (Kinect) to capture environmental point clouds at rate of 30 Hz. The FoundationPose model serves as object pose estimation module, which processes the point clouds from the RGB-D camera to predict the 6D object pose, achieving a 93.22% ADD AUC score the YCBInEOAT dataset at 30 FPS. By sampling 4,096 points from the object's mesh and applying the estimated pose from FoundationPose, we obtain cleaned object point clouds for the past 5 frames (To 5).
Figure Additional Visualization of Rerl-World Experiments on Robotic Handover Tasks.
Our algorithm takes these cleaned object point clouds along with the shadow hand poses from the previous 5 frames as input conditions, and predicts the hand's poses for the next frames (T, 3). These predicted poses are executed directly on the physical robot, enabling fully end-to-end action generation system. terms of system performance, the perception module operates at approximately 30 FPS, while the action prediction module achieves over 70 FPS with single iteration setting. The
25

---

complete end-to-end system maintains a comprehensive operating rate of approximately 25 FPS, meeting real-time interaction requirements. Figure 6 shows the results of two additional sets of handover tasks.
F
Discussion
E1
Further Discussion on Vision-Languagg-Actionn (VLA) Models
In the main sections of this paper, we have detailed and validated the superior performance of FregPolicy in learning specific robotic manipulation tasks. Its innovative frequency-domain autore- gressive mechanism and the use of continuous tokens have demonstrated significant advantages in both precision and efficiency for single-task learning. However, a natural and promising extension is to investigate the adaptability of FreqPolicy in more complex and generalized multitask learning scenarios, especially when task instructions are given in natural language. Vision-Language-Actii (VLA) models provide a powerful framework for achieving such language-driven multitask robot control.
Therefore, this section aims to conduct a preliminary exploration of FreqPolicy's 's potential when applied to VLA models. Given that policy learning in multitask environments is still a multifaceted and challenging research problem, we do not aim to provide a solution here. Instead, our goal is to whether
assess FreqPolicy the core ideas of FregPolicy, such as frequency-domal decomposition and hierarchical learning, can benefit VLA models. We also outline possible directions for future research. We believe that this discussion can help to provide a more comprehensive understanding of FreqPolicy's potential and its future development.
multitask benchmark and to provide forward-looking discussion
Multitask Benchmark. To preliminarily assess FreqPolicy's s performance multitask setting, we selected RoboCasa 2 as the benchmark platform. RoboCasa comprises a suite of tasks defined within a simulated kitchen environment, representing complex interaction scenarios that robots might encounter in the real world. In this exploratory experiment, we focused on 24 "atomic" tasks, which cover fundamental sensorimotor skills such as pick-and and--lacee, opening and closing doors, pressing buttons, and turning faucets. Placing our method in such simulated environment, which possesses certain level of difficulty and a rich variety of tasks, helps us to preliminarily understand its potential and areas for exploration in multitask learning.
Baseline. For effective comparison, we first selected the Diffusion Policy and the autoregressive method BC-Transforma [25] implemented in RoboCasa as direct baselines for our FreqPolicy. Considering the characteristies of VLA models, we further introduced GROOT-NI [3] as a reference. GROOT-NI is VLA model with tightly coupled dual-system, where its vision-language module responsible for understanding the environment and instructions, and the subsequent Diffusion Transformer module employs Flow Matching technology to generate smooth action sequences in real-time. Selecting these baselines helps us to more clearly position the relative performance of FreqPolicy within existing VLA frameworks.
Implementation Details. In this initial exploration, our FreqPolicy model is directly integrated on top of the Diffusion Policy framework within RoboCasa. To ensure fair comparison, the main parameters and observation inputs used by FreqPolicy are kept consistent with those of Diffusion Policy. This simplified integration aims to quickly validate the basic adaptability of Freq core mechanisms a multitask scenario, rather than to perform deep customization and optimization.
Results and Discussion. The experimental results, as shown in Table TT offer an initial insight into FreqPolicy's 's performance on the RoboCasa multitask benchmark. The data indicate that, compared to Diffussin Policy and BC-Transformer our method demonstrates certain ad advantage in overall multitask success rates. This preliminarily suggests that FreqPolicy's frequency-domain processing mechanism, and its hierarchical modeling approach to action sequences, are not only effective for single-task learning but may also bring positive impacts scenarios requiring the simultaneous handling of multiple tasks. However, when compared to the GROOT-N1 model, which is specifically designed for VLA, FreqPolicy's is current performance still shows gap in multi-task success rates. We attribute this primarily to the fact that FreqPolicy, in its current design and similar to Diffusion Policy, focuses more on action generation and optimization. It has not been specifically enhanced for deep understanding of complex language instructions and multi-modal scene perception to
26

---

Table 11: Multitask results on RoboCasa. Experimental results of BC-Transformer, Diffusion Policy and GROOT-NI are from the GROOT-NI paper.
BC- -C-Trrnsformer
Diffusion Policy GROOT-N1
FreqPolicc(OOur)
Success Rate
26.3%
25.6%
32.1%
27.4%
the same extent GROOT-NI. One of the strengths of VLA models lies in their powerful semantic understanding and scene perception capabilities, enabling them to generalize better to unseen instruction and environment combinations.
Nevertheless, these preliminary results also provide us with important insights: could FreqPolicy's unique frequency-domat analysis and modeling approach serve as a beneficial supplement when integrated into more powerful VLA frameworks? For instance, FreqPolicy's s ability to capture the smoothness and structural information of action signals might assist VLA models in generating more stable and physically plausible action sequences. Exploring how to effectively combine the rrequency-doman strengths of FregPolicy with the semantie understanding capabilities of VLA models, with the aim of further enhancing overall multi-task learning performance, will be a highly valuable research direction for us in the future. This might involve designing new fusion mechanisms or tailoring FreqPolicy's 's frequency decomposition strategies and autoregressive processes to the specific characteristics of VLA tasks. In summary, while FreqPolicy was not natively designed for VLA tasks, its core ideas demonstrate potential worthy of further investigation in the broader field of multi-task robot learning.
F.2 Discussion Frequency Domain
In the preceding discussions, we have preliminarily shown that the action signals in robotic ma- nipulation tasks exhibit significant compressibility in the frequency domain, with most critical information concentrated in the lower frequency bands. To make this statement precise, we quantify "compressibility" by the energy proportion:
E(p)
|x
1X&l
Le. the share of total signal energy that is preserved when only the lowest p% DCT coefficients retained (see Figure
This section aims to provide a more in-depth discussion and analysis of the characteristics of action signals, based on a broader set of tasks and more detailed visualizations (as shown in Figures to 30). These supplementary figures provide action visualizations (a), frequency band energy heatmaps for each action dimension (b), and success rate curves for actions reconstructed with varying frequency ratios (c) for each task, thereby offering more robust support for our core arguments.
High-Ding Dimensional Tasks. In high-dimensional action spaces (22 dimensions) within Dexart tasks (Figures 7 Dexart Bucket, Faucet, Laptop and Toilet), we observe consistent trends. The success rate curves (c) for these tasks generally show that even using only 30%-70% of the low-frequency components is often sufficient reconstruct action sequences capable of task completion, strongly supporting the core hypothesis that high-frequency components contribute relatively little to the macroscopic success of these complex tasks. Concurrently, energy heatmaps (b) clearly demonstrate that different action dimensions exhibit varying dependencies on frequency components; some dimensions (like large-range arm movements) have energy highly concentrated in very low-frequency bands, while others (like fine finger postures) might retain significant energy in relatively higher bands. For instance, Dexart Faucet (Figure (8p), energy distribution in the 10-30% or even higher frequency bands for some dimensions might correspond to fine adjustments for turning faucet. Although overall trends are similar, the minimum frequency ratio for high success rates varies slightly across Dexart tasks, with Dexart Laptop (Figure(pb) for example, reaching a success plateau around 0.4-0.6 frequency ratio, suggesting subtle differences in action signal fidelity requirements for various complex manipulations.
now-Dimensional Tasks, Compared to high-dimensional Dexart tasks, low-dimensional Meta- World tasks (Figures
II 30
typically 4-dimensional action spaces) exhibit more pronounced
27

---

low-frequency dominance. In most Meta- World tasks, success rate curves (c) indicate that only 10%- 40% of the low-frequency ratio is sufficient for near-perfect task success, with tasks like Meta-World World Coffee-Pull (Figure 13c) and Meta-World Disassemble (Figure 15p) requiring only about 20% low- frequency signal. This suggests higher compressibility action signals for these simpler robotic tasks, corroborated by their energy heatmaps (b) where most action dimensions show energy concentrated the lowest 0-10% band, consistent with their typically smoother, direct motion trajectories. Minor exceptions. such Meta-World Shelf- -lace (Figure 28c) or Meta-World Push-Wall (Figure (26-), might need slightly more frequency components due to potentially higher precision demands at the end-effector.
Discussion. Synthesizing these analyses, we discover the universality and variability of frequency compression across task types: action signals are generally compressible in both high-dimensional complex and low-dimensional structured tasks, though the required frequency bandwidth varies with task dimensionality, complexity, and operational specifics. The heterogeneity of action dimensions highlighted by heatmap analysis suggests that future work on dimension -nneesioo-adaptive frequency pro- cessing, rather than global uniform cutoffs, could be promising optimization, such dynamically allocating frequency components based on each dimension's 's energy to balance performance and efficiency. A deeper understanding of task action frequency characteristics can also guide policy learning algorithm design, for example, by using low-frequency biases or stronger regularization for low-frequency dominated tasks to accelerate learning and enhance generalization.
In conclusion, the additional frequency domain analysis in this section provides more comprehensive empirical support for FreqPolicy's s core mechanisms, points to valuable directions for optimizing frequency-basee based robot learning, deepens our understanding of robotic action nature, and underscores the potential of the rreuuecy-domanl perspective in building efficient, robust robotic agents.
28

---

(Action Dimension: 22
(8)
P
0:36
3.63
10-
0:48
0.80
1.00
2.00
3.60
8.85
0.5
Action Dimunsin
1.00
5.00
3.60
8.85
Inargg Rate
1.00
2:00
8.55
3.20
8.55
Sucoes
1.28
(4)
0:48
2.01
3.60
0.33
2.03
3.60
8.50
3.20
5.00
3.60
8.86
0.39
1.00
3.00
3.05
330%
10-30%
10:00:
50-300%
Frequency Domsin Scale
Frequescy Band
Vintalicttion
Frequccc map different dimemsium
frequency
Figure 7: Frequency Domain Analysis Dexart Bucket.
Task Name: Dexart Faucel (Action Dimsensio:: 22)
8.41
4203
0:32
6101
1.0
0.04
0.13
0.30
5.00
0.99
08.33
0.30
5.0
Aption Dimasinn
0.99
0.83
0.00
Sucess Rite
5.00
0.33
11:34
5.00
Emerg
0.99
01:3
0.30
1.00
0.99
0.23
0.30
8.00
0.96
01:14
8.00
0.31
0.23
5.00
8.00
0.0
0.16%
13:309
Fiqqunny Sand
52-1095
0.0
0.2
Frequency Domain Scale
0.6
0.5
0.5
1.0
(4) Viuulization
Frequency map
(c) Success netiooas by Erequency
Figure 8: Frequency Domain Analysis of Dexart Faucet.
Task Name: Dexart (Action Dimension: 22)
2.3
11C
6113
$.13
1.0
1:00
2.00
3.50
9.80
2.18
8.03
9.80
0.5
Artien Disvasion
8.87
2.40
0.00
Lourg Ratio
Sacteso Raie
1.09
0.
0.37
1:08
2.00
8.33
2.40
0.03
8.
0.45
1.65
8.60
0.60
02-
1.00
2.80
3.10
0.
0.23
3.60
0.0
6-40%
1,02
10-32%
2.00
15-505
60
0.2
Frequency Damain Scale
0.6
Action Visualization
(b) Prequeacy band beat map of different action dimennions
(4) Success rade afaciions by different frequency ratios
Figure 9: Frequency Domain Analysis of Dexart Laptop.
Dimension:
[:31
(
(
0:21
0:01
6m1
0.59
0.22
6.00
5.00
5.00
(13)
5.0
Acrion Dimmnson
0.53
0.53
Rote
0.00
0.31
0.6
8.48
0131
Eowng
0.57
0222
0.00
5.00
04
(4)
0.84
0.33
0601
0.99
01:11
0151
0.7
0.50
0.50
0.03
1.00
01:2
0151
5.00
0.22%
10 30%
3030%
39.000%
Frequency Domain Scale
(a) Action Vinralization
Friquency hand beat of different sction dimemsion
(6) Success mate of sctions by difffrront frequency ratios
Figure 10: Frequency Domain Analysis of Dexart Toilet.
29

---

Task Meta-Workd
Dimeusion:
(1)
121
0.11
535
0.31
0.5
Acrisn Dimanstin
Sucess Rall
0.6
Eowngy
0.12
0.87
02
1.99%
15%
50
Frequency Domzin Scale
(A) Acttci Vicualization
(b) Prequency band beat map of different oction dimensions
(c) Success rate of actioas by different trequency rotios
Figure 11: Frequency Domain Analysis of Meta- World Bin-Picking.
Task Name: Meta-Workk World Box-Close (Action Dimension:
AR
Action Dimuniionn
Stocess Rase
6:91
646
6.1
2.14
04
0.0
04
Prequency Domain Scale
(4) Vaualication
Erequency map of
(e
frequency ratios
Figure 12: Frequency Domain Analysis Meta- World Box-Close.
Task Name: Meta-World Coffee-Pull (Action Dimension:
8:30
6.11
Astion Dimersion
Saccess Rtae
8:05
0.44
6.1
6.10%
30%
3000%
50-200%
Froquency
(a) Action Visualiration
(b) Frequency band beat map of different action dimensions
(6) Suceom rale of actions by differeet frequency ratios
Figure 13: Frequency Domain Analysis of Meta-World Coffee-Pull.
Name: Meta-World Coffee-Pash (Action Dimension: 4)
(1)
(2)
0.77
2.5
0.31
Action Dimendon
Success Rdic
3.31
0.6
(3)
(4)
0.2
0.14
012
0.00
0.0
0.0
0.4
1.0
Frequency Domain Scale
Requency Sand
Action Vicualization
dimensions
of actioas different
Figure 14: Frequency Domain Analysis of Meta- World Coffee-Push
30

---

Task Name: Meta-World Disassemble Dimension:
12)
2,00
8:23
0.2
Action Dimaniion
Saooess Itle
06
E-Q
8:11
02
6-10%
100%
90
Frocuency Domain Scale
(a) Action Visualization
Frequency band beat map of different action dimensioas
(0 Success rate of actions by dificrest frequency ratios
Figure 15: Frequency Domain Analysis of Meta- World Disassemble.
Meta-World
Task Name: Meta- World Drawer-Open (Action Dimension: 4)
(2)
90
Action Dimersien
Success Rate
8:08
0.22
0.33
1.9
06
(3)
(4)
1.00
0.0
0.2
0.1
0.6
1.0
Froquency Domannn Scale
Freg unary Bost
Frequrccy map of
(c)
frequency
Figure 16: Frequency Domain Analysis Meta- World Drawer-Open.
Task Name: Metn-World Hammer (Action Dimension:
10
0.36
Aztion Dimeraim
Swoces Ri12
0,07
3.00
0.00
0.00
3-20%
13.3%%
12.53%
50 50.00%
Frequency
(5) Action Viszalization
(b) Prequency hand beat map of different action dimunsion
(6) Sncrees rato of actions by different froquency ratlos
Figure 17: Frequency Domain Analysis of Meta-World Hammer.
Name: Meta- Hand-Insert (Action Dimension: 4)
(8)
(2)
0.33
0.01
Antie Dimunsin
Suctss Rtle
010
0.6
(3)
(4)
0.33
0.00
0.00
0.0
0.0
0.0
0.4
Frequency Domain Scale
Fresuecy Band
Actien Visnalization
Frsquency
map dimession
froquenccy
Figure 18: Frequency Domain Analysis of Meta- World Hand-Insert.
31

---

Task Mela-World Handke-Press (Action Dimension:
121
0/00
0.5
Apiion Dimension
Sucess Rill
0.6
Eowngy
0.14
02
$.99%
59%
13.95
90
Frequency Domain Scale
(a) Actici Visualization
(b) Prequency band beat map of different oction dimensions
(c) Success rute of actioas by different trequency rotios
Figure 19: Frequency Domain Analysis of
Wandle-Press,
Task Name: Meta- World Lever-Pull (Action Dimension:
(2)
Action Dimersiio
Sucoes Male
0.08
0.00
0.00
0.04
0.6
(3)
(4)
1:00
0.0
0.2
0.4
0.5
0.5
1.0
Frequency Domsin Scale
Pretuency Band
Prequency map
frequency
Figure 20: Frequency Domain Analysis Meta- World Lever-Pull.
Task Name: Metn-World Peg-Insert--oe Side (Action Dimension:
0.88
060
Astion Dimension
Saacess Raae
0.11
0.04
0.0
640
00
3.50%
12:39h
33-00%
50-10m
Frequency
(3) Artion Visnaliration
(b) Frequency band beat map of different action dimensions
(6) Sucoom rale of actions by different frequency ratios
Figure 21: Frequency Domain Analysis of Meta- Meta-World Peg-Insert-Side.
Name: Meta-World Pog-U Uinpupg-ppppdee (Action Dimension: 4)
(8)
(2)
0.AE
Action Dimuniion
Sacces Dtae
0.32
000
(4)
0.14
0.0
002
00
0.0
0.4
0.6
1.b
Frequency Docaain Scale
Fresuency
Frequrncy map dimensions
frequency
Figure 22: Frequency Domain Analysis of Meta- World
32

---

Task Meta-Workd Plek-OuttOOfBooke (Action Dimension:
(2)
0.17
0.4
Aciion Dimantion
Saooess Itle
06
0.43
044
0.
$.999
13.39%
59%
109
90
Frocuency Domain Scale
(a) Action Visualization
Frequency band beat map of different action dimensions
(e Saccess rate of actions by dificrent frequency ratios
Figure 23: Frequency Domain Analysis of Meta- World Pfck-Out-Of-Hole.
Task Name: Meta-Worid Plck- Place-Place- Place-Wall (Action Dimensiom: 4)
(2)
Action Diconsioo
Sucoes Male
9:51
1.5
0.01
0.00
0.6
(3)
(4)
0.
0.0
0.2
0.5
0.5
0.5
1.0
Frequency Domsin Scale
Treguncyy Fand
Prequency map
frequency
Figure 24: Frequency Domain Analysis of Meta- World Pick-Place- Wall.
Task Name: Meta- World Pick-Place (Action Dimension:
040
Asiinn Dimension
Saccess Raae
0.33
0.08
0.0
0.00
9-20%
1333%
30.50%
56330%
Froquency
(4) Artion Visualization
(b) Frequrncy band beat map of different action dimensions
(6) Saceom rale of actions by diffirrent frequency ratios
Figure 25: Frequency Domain Analysis of Meta World Pick-Place.
Name: Mets-World (Action Dimension:
())
(2)
Aprion Dimonson
Success Rdic
0.34
0.10
0.6
(4)
0.2
0.33
0.00
0.00
0.0
0.0
0.4
0.6
1.0
Frequency Domain Scale
Frecuecy
Action Visualization
Frsquency map different dimession
froquencc
Figure 26: Frequency Domain Analysis of Meta- World Push-Wall.
33

---

Task Meta-Workd
Dimension:
(1)
221
0.32
0.31
0.5
Acrion Dimansiin
Sucess Rill
0.6
Eowngy
0.11
282
0:11
02
1.99%
15%
12.55
100%
50
Frequency Domzin Scale
(a) Action Visualization
(b) Prequency band best map of different oction dimensions
(c) Success rute of actions by different trequency ratios
Figure 27: Frequency Domain Analysis of Meta-Worde Push.
Task Name: Meta- World Shelf-Place (Action Dimension:
Action Dimmmiioo
Sucoes Male
0.88
3144
0.33
0.00
0.6
(3)
5:30
0.
0.0
0.2
0.4
0.5
0.5
1.0
Frequency Domsin Scale
Prepency fand
Prequency map
frequency
Figure 28: Frequency Domain Analysis of Meta- World Shelf-Place.
Task Name: Meta World Stick-Pull (Action Dimension:
0.31
Aztion Dimersim
Success Rade:
0.38
301
3.00
0.00
3-20%
18.00%
13.53%
5o 1000
Frequency
(a) Action Viszaliration
(b) Prequency hand beat map of different action dimussions
(g) Sncrees rale of actions by different froquency ratlos
Figure 29: Frequency Domain Analysis of Meta-World Stick-Pull.
Name: Meta-World Sweep (Action Dimension:
())
(2)
0.35
0.01
Aption Dimonsinn
Sucsss Rtle
1.80
0.6
(3)
(4)
0.77
000
0.0
0.0
0.4
Frequency Domain Scale
Precuarcy Band
Action Vionatization
Frsquencc
map dimession
froquencc
Figure 30: Frequency Domain Analysis of Meta-Worke Sweep.
34


Reviews:
[{"review_id": "fzerj8RigV", "reviewer": "Reviewer_ByCW", "summary": "The paper proposes FreqPolicy, a new approach to teaching robots how to move by analyzing actions through their frequency components. The key insight is that robotic movements have natural structure - broad, overall patterns appear in low frequencies while precise, detailed movements show up in high frequencies.\nThe method works by breaking down actions into these frequency layers and learning them step-by-step in a hierarchical manner. Instead of forcing actions into discrete categories, FreqPolicy employs smooth, continuous representations, combined with a diffusion-based system, to generate more natural and precise movements.\nThe main innovations include creating a frequency-based framework that captures the inherent structure of how robots move, developing a continuous representation system that avoids the limitations of categorizing actions into fixed bins, and demonstrating superior performance on robot manipulation tasks with better accuracy and faster computation compared to existing approaches.\nEssentially, by considering robot actions like how we think about music or signals, with different frequency components conveying distinct types of information, this method achieves more effective and efficient robot learning.", "questions": "1.  How does FreqPolicy handle the trade-off between computational efficiency and action precision, especially given that high-frequency components encode fine local details?\n2.  What are the specific mechanisms or architectural components within FreqPolicy that enable the \"coarse-to-fine\" generation process for hierarchical frequency components?\n3.  The paper mentions real-world experiments with only 1 iteration for efficiency. How does this single-iteration performance compare to higher iteration counts in simulated environments, and what are the implications for deploying FreqPolicy in other real-time robotic applications?", "limitations": "Please check the above weakness section", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n  * **Originality:** FreqPolicy introduces a novel and well-motivated approach by representing actions in the frequency domain, addressing limitations of existing methods. The progressive modeling with continuous latent representations is a unique contribution.\n  * **Significance:** It significantly advances robotic manipulation by achieving state-of-the-art performance and faster inference, making it highly practical for real-time applications. The analysis of task-dependent frequency requirements is a valuable insight.\n  * **Quality:** Extensive experiments across diverse benchmarks consistently demonstrate superior accuracy and computational efficiency. The frequency domain analysis empirically justifies the core idea.\n  * **Clarity:** The paper is generally well-written, clearly explaining the problem, method, and results with effective figures.\n\n**Weaknesses:**\n\n  * **Clarity - Specifics on Diffusion-Based Decoding:** More detail on the diffusion-based decoding and its integration in the main text would improve understanding.\n  * **Clarity - Masked Generative Strategy:** A slightly more detailed explanation of its benefits (cost reduction, diversity) beyond mask ratios is needed.\n  * **Quality - Ablation on Frequency Levels:** The paper lacks explicit discussion or ablation studies on the selection and progression of frequency levels for autoregressive iterations.\n  * **Limitations Section:** A dedicated limitations section in the main paper would be beneficial for critical self-reflection.", "was_revised": true, "final_justification": "Thank you for your comprehensive rebuttal. The additional experiments and explanations address my earlier concerns and enhance the quality of the work.", "rebuttal": "We thank the reviewer for the valuable comments which help improve the paper.\n\n**W1.Clarity - Specifics on Diffusion-Based Decoding: More detail on the diffusion-based decoding and its integration in the main text would improve understanding.**\n\nWe will add detailed explanation of the diffusion-based decoding mechanism in the main text:\n\n  - **Diffusion Decoding Mechanism:** As shown in Figure 2, FreqPolicy aims to generate actions containing full-frequency information from action tokens at any given frequency level (controlled by index tokens).\n\n  - **Specific Training Process:** At index=k, the observation features and corresponding k-level reconstructed actions $y^k$ are input to FreqPolicy's MAE to obtain latent representation $z^k$. $z^k$ serves as the condition for the diffusion model, guiding the generation of full-frequency actions from noise sampling.\n\n  - **Iterative Inference:** As illustrated in Figure 2(b), the model starts reconstructing full-frequency actions from low-frequency levels. Through DCT/iDCT operations, it obtains actions containing higher frequency information for the next iteration, achieving progressive action generation from coarse-grained (low-frequency) to fine-grained (high-frequency).\n\nConcrete examples can be found in our response to Reviewer Zaqb's weakness 1.\n\n**W2.Clarity - Masked Generative Strategy: A slightly more detailed explanation of its benefits (cost reduction, diversity) beyond mask ratios is needed.**\n\nWe provide additional experimental evidence on the specific benefits of our masking strategy:\n\n| Epoch  | 0 | 400 | 600 | 800 | 1000 | 1200 | 1400 | 1600 | 1800 | 2000 | 2200 | 2400 | 2600 | 2800 |\n|:------:|:-:|:---:|:---:|:---:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n| Mask   | 0 | 25  | 42  | 54  | 57   | 65   | 67   | 71   | 74   | 69   | 75   | 73   | 68   | 72   |\n| No Mask| 0 | 4   | 7   | 27  | 30   | 32   | 34   | 42   | 52   | 54   | 64   | 67   | 65   | 71   |\n\n  - **Training Efficiency Improvement:** As shown in the table below, in the average success rate comparison across all tasks in the Adroit benchmark, following DP3 testing protocol with 20 random environments, the masked approach converges faster and achieves better performance:\n\n  - **Diversity Enhancement:** The high average success rates across 20 random environments demonstrate that the masking strategy enhances the model's generalization ability and action diversity across different environments.\n\n**W3.Quality - Ablation on Frequency Levels: The paper lacks explicit discussion or ablation studies on the selection and progression of frequency levels for autoregressive iterations.**\n\nWe have conducted ablation studies on frequency level selection for autoregressive iterations in the main text, with results shown in Figure 4. For an action chunk of length T, DCT transformation yields T frequency coefficients. Our key parameter is the number of autoregressive iterations $N_{iter}$, where the frequency spectrum is automatically divided into $N_{iter}$ equal-interval frequency bands, and the model processes these frequency bands sequentially from low-frequency to high-frequency components.\n\n**W4.Limitations Section: A dedicated limitations section in the main paper would be beneficial for critical self-reflection.**\n\nDue to space constraints in the main text, we prioritized presenting the core methodology and experimental results, placing the limitations analysis (detailed in Appendix G) in the appendix. We completely agree with your perspective that this content deserves a more prominent position in the main paper. Therefore, when preparing the final camera-ready version, we will move this discussion to the conclusion section of the main text for clearer self-reflection and future work prospects.\n\n**Q1.How does FreqPolicy handle the trade-off between computational efficiency and action precision, especially given that high-frequency components encode fine local details?**\n\nFreqPolicy handles the trade-off between computational efficiency and action precision directly and flexibly through the number of hierarchical autoregressive iterations $N_{iter}$. Our experimental results (as shown in Figure 4) clearly reveal this: as the number of iterations $N_{iter}$ increases, the model's inference time grows correspondingly, but the task success rate does not necessarily increase. We found that when excessively subdividing and iterating over high-frequency noise, high-frequency data tends to fall into long-tail distributions with less effective signal information, potentially introducing noise instead. In our experiments, $N_{iter}=4$ achieves the optimal balance between performance and efficiency across multiple tasks.\n\n**Q2.What are the specific mechanisms or architectural components within FreqPolicy that enable the \"coarse-to-fine\" generation process for hierarchical frequency components?**\n\nFreqPolicy's \"coarse-to-fine\" generation process is achieved through several key architectural components working collaboratively within an iterative autoregressive framework:\n\n  - **DCT/IDCT Transformations:** DCT decomposes action sequences into frequency components, separating \"coarse\" low-frequency information representing overall motion contours from \"fine\" high-frequency information for precise local adjustments.\n\n  - **Hierarchical Frequency Autoregressive Iteration:** The generation process is structured as a for-loop (see Appendix Algorithm 2), strictly traversing frequencies from low to high. In the k-th iteration (assuming k iterations divide into k frequency bands):\n    - Low-frequency extraction extracts k-1 level low-frequency actions $y^{(k-1)}$ from the previous prediction's full-frequency action $\\hat{x}^{(k-1)}$ through IDCT\n    - Conditional encoding inputs $y^{(k-1)}$ and robot observations O as conditions to FreqPolicy encoder/decoder, obtaining latent embedding $z^{(k-1)}$\n    - Diffusion generation uses $z^{(k-1)}$ as condition input to the diffusion model, generating complete actions $\\hat{x}^k$ containing all frequency details\n\n  - **Core Mechanism:** At each stage, the model revisits and optimizes the complete action sequence based on increasingly frequency-level and refined actions, enabling progressive generation from coarse-grained to fine-grained.\n\n**Q3.The paper mentions real-world experiments with only 1 iteration for efficiency. How does this single-iteration performance compare to higher iteration counts in simulated environments, and what are the implications for deploying FreqPolicy in other real-time robotic applications?**\n\nIn our real-world dynamic object grasping experiments, we indeed set the iteration number $N_{iter}=1$, primarily determined by the extreme real-time requirements of the task itself. This task involves capturing a moving object, imposing extremely stringent latency requirements on the perception and control loop.\n\n  - **Why Single Iteration:** In this scenario, inference speed is the primary factor determining success. Our FreqPolicy achieves action inference speeds of up to 70 FPS at $N_{iter}=1$, while the baseline method DP3 only reaches 25 FPS. Considering that the perception module also consumes time, only our method can meet such stringent end-to-end latency requirements to successfully complete dynamic grasping. If multiple iterations were used, the FPS drop would prevent the robot from catching up with the moving object.\n\n  - **Consistency with Simulation Results:** This real-world performance is completely consistent with results observed in our simulation environments (as shown in Figure 4). Figure 4 clearly demonstrates that $N_{iter}=1$ corresponds to the highest computational efficiency and shortest inference time configuration. While increasing iteration counts in simulation can further improve success rates for static tasks, single iteration already provides a very strong and efficient performance baseline.\n\n  - **Deployment Flexibility:** FreqPolicy's core value lies in flexibly balancing efficiency and precision according to task requirements:\n    - High real-time tasks: $N_{iter}=1$ maximizes inference speed (e.g., dynamic grasping)\n    - High precision tasks: Appropriately increase iteration counts (e.g., $N_{iter}=4$ for Adroit benchmark)\n    - Real-world static tasks: Can increase iteration counts, FPS drops to 20-30 but success rate improves\n\nThis adaptability makes FreqPolicy suitable for a wide range of real-time robotic application scenarios."}, {"review_id": "iGs7ANgsx5", "reviewer": "Reviewer_wtRU", "summary": "This paper introduces FreqPolicy, a novel visuomotor policy learning framework that represents robot actions in the frequency domain and generates continuous-action trajectories via a hierarchical, autoregressive diffusion model. By decomposing trajectories into low-to-high frequency components using DCT and mapping them into continuous latent tokens, FreqPolicy progressively refines coarse global motions into fine local details. Extensive simulation results on Adroit, DexArt, Meta-World and RoboTwin benchmarks demonstrate state-of-the-art success rates and a favorable inference–performance trade-off.", "questions": "1. Can you provide concrete success‐rate and robustness metrics from your “real-world” experiments rather than only qualitative snapshots?\n\n2. How does FreqPolicy compare against other frequency‐aware action tokenizers such as FAST+DP or DP3 on your core benchmarks?\n\n3. Because of marginal success‐rate gains, what additional evaluation metrics or experiments (e.g., trajectory smoothness in the frequency spectrum, robustness to noise, sample efficiency, or control bandwidth analysis) would you recommend to more convincingly validate the benefits of a frequency-domain approach?", "limitations": "Please see the weaknesses. If you add more experiments to solve my concerns, I would like to raise my scores.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Frequency-Domain Representation: Modeling actions via DCT naturally separates global versus fine-grained motion, yielding smoother low-frequency priors and targeted high-frequency refinements—a clear conceptual advance over purely time-domain AR or diffusion approaches.\n\n2. Rich Simulation Evaluation: The authors benchmark on 48 simulation tasks across multiple datasets, outperforming strong baselines (e.g., DP3, Mamba Policy) on 8 of 10 Adroit tasks and averaging 67.9 % success versus 65.8 % from competing methods.\n\n3. Efficiency & Real-Time Potential: FreqPolicy’s single-iteration inference runs at 0.21 s per trajectory (≈ 70 FPS) and demonstrates a qualitative handover on real hardware, highlighting its suitability for latency-sensitive control.\n\nWeaknesses:\n\n1. Lack of Quantitative Real-Robot Metrics: The “real-world” section reports only inference speed and a qualitative handover snapshot, without reporting success rates or robustness on the physical platform.\n\n2. Missing Baseline with Frequency-Tokenizers: Despite the frequency focus, there is no comparison against other frequency-aware action tokenizers such as FAST + DP or DP3 variants that explicitly compress frequency components.\n\n3. Marginal Success-Rate Improvements: Although FreqPolicy outperforms baselines, the absolute gains in success rate are modest (e.g., from 65.8 % to 67.9 % on average), raising questions about the practical significance of the frequency-domain representation.", "was_revised": true, "final_justification": "Thank you for the thorough rebuttal. The additional experiments address my previous concerns about validation and significantly strengthen the manuscript. Accordingly, I am happy to raise my overall score.", "rebuttal": "Thank you for your valuable feedback, which helped us significantly improve our experimental validation and comparative analysis.\n\n**W1 or Q1:**\n\nWe conducted 10 comparative experiments on real-world dynamic handover tasks with stringent real-time requirements.\n\n- **Experimental Setup:** Dynamic object handover task requires the robot to track and grasp moving objects in real-time, imposing extremely stringent latency requirements on the perception-control loop.\n\n- **Quantitative Results:**\n\n    | Task | DP | DP3 | Ours |\n    |:------:|:--:|:---:|:----:|\n    | Handover | 0 | 0 | 40 |\n\n- **Performance Analysis:**\n  - **DP and DP3:** Due to inference speed limitations (<25 FPS), they cannot meet the real-time requirements of dynamic tasks, resulting in complete failure.\n  - **Our Method:** Using 1 iteration, achieves inference speed of 70 FPS and completes the task with 40% success rate. Demo videos are included in the submitted supplementary materials.\n\n**W2 or Q2:**\n\n- **Method Route Clarification:** We and FAST adopt two different frequency domain processing approaches:\n  - **FAST Route:** Uses quantization to filter high-frequency components, introducing discretization losses.\n  - **Our Route:** Different tasks need different frequency levels - high-DOF tasks need full-frequency information. We use continuous tokens to preserve action space properties.\n\nFollowing the reviewer's suggestion, we conducted detailed comparisons with FAST+DP3 methods.\n\n- **Experimental Setup:** In addition to FAST+Diff method, we also tried FAST+AR as an experimental baseline.\n\n- **Experimental Results Comparison (Success %):**\n\n    | Method | Hammer | Door | Pen | Pick Out of Hole | Soccer | Stick Pull | Avg |\n    |:------:|:------:|:----:|:---:|:----------------:|:------:|:----------:|:---:|\n    | FAST+Diff | 13 | 18 | 16 | 32 | 26 | 50 | 26 |\n    | FAST+AR | 0 | 0 | 0 | 0 | 0 | 0 | 0 |\n    | Ours | 100 | 65 | 59 | 30 | 32 | 62 | 58 |\n\n- **FAST Reconstruction Error Analysis (RMSE):**\n\n    | Task | Hammer | Door | Pen | Pick Out of Hole | Soccer | Stick Pull |\n    |:----:|:------:|:----:|:---:|:---------:|:------:|:----------:|\n    | FAST_RMSE | 0.59 | 0.57 | 0.63 | 0.26 | 0.23 | 0.32 |\n\n- **Detailed Analysis:**\n  - **FAST+AR Failure Reason:** Due to token length mismatch issues in FAST's decoder, resulting in all-zero outputs. FAST authors acknowledge this problem in their official repository and suggest using π_0 diffusion decoding as a solution.\n\n  - **FAST+Diff Performance Analysis:** From RMSE data, reconstruction errors between FAST-processed actions and ground truth are significantly higher in high-DOF tasks (Hammer: 0.59, Door: 0.57, Pen: 0.63) compared to low-DOF tasks (Pick_hole: 0.26, Soccer: 0.23, Stick pull: 0.32). We believe this is the main factor causing performance degradation due to FAST filtering out partial frequency domains and using quantization operations.\n\n  - **Task Characteristic Impact:** The data further validates our viewpoint that different tasks require different frequency domain levels - high-DOF tasks are more affected by FAST processing, while low-DOF tasks are relatively less affected but still show degradation.\n\n**W3 or Q3:**\n\n**1.Success Rate Analysis:** We achieve significantly higher success rates on high-DOF dexterous manipulation tasks. As future demands for high-DOF dexterous operations grow, the advantages of our modeling approach will become more apparent. For example, in Adroit and DexArt benchmarks, our method shows significant advantages on complex manipulation tasks. Additionally, we can efficiently complete tasks under different inference speed requirements by adjusting iteration counts.\n\n**2.Trajectory Smoothness Analysis:** We evaluate trajectory quality through three metrics:\n\nHigh-frequency energy ratio (smaller = smoother):\n\n$$\\text{ratio} = \\frac{1}{D} \\sum_{d=1}^{D} \\frac{\\sum_{f \\in F_{\\text{high}}} |X_d(f)|^2}{\\sum_f |X_d(f)|^2}$$\n\nRMS gradient (smaller = gentler changes):\n\n$$\\text{RMS}\\_{\\text{grad}} = \\frac{1}{D} \\sum_{d=1}^{D} \\sqrt{\\frac{1}{N-1} \\sum_{t=1}^{N-1} (a_{t+1,d} - a_{t,d})^2}$$\n\nRMS curvature (smaller = more uniform):\n\n$$\\text{RMS}\\_{\\text{curv}} = \\frac{1}{D} \\sum_{d=1}^{D} \\sqrt{\\frac{1}{N-2} \\sum_{t=1}^{N-2} [(a_{t+2,d} - a_{t+1,d}) - (a_{t+1,d} - a_{t,d})]^2}$$\n\nWhere $N$ is time steps, $D$ is action dimension, $a_{t,d}$ is action value, $X_d(f)$ is frequency representation, and $F_{\\text{high}}$ denotes high-frequency components.\n\n| Method/Task | Hammer | Door | Pen | Pick Out of Hole | Soccer | Stick Pull | **Average** |\n|:-----------:|:------:|:----:|:---:|:----------------:|:------:|:----------:|:-----------:|\n| **DP3** | 0.067/0.174/0.256 | 0.094/0.202/0.263 | 0.054/0.144/0.187 | 0.118/0.422/0.593 | 0.043/0.079/0.096 | 0.080/0.472/0.681 | 0.076/0.249/0.346 |\n| **Mamba** | 0.066/0.175/0.261 | 0.096/0.209/0.265 | 0.057/0.147/0.188 | 0.097/0.424/0.591 | 0.049/0.089/0.114 | 0.109/0.508/0.721 | 0.079/0.259/0.357 |\n| **Ours** | 0.065/0.171/0.252 | 0.096/0.213/0.267 | 0.051/0.137/0.174 | 0.101/0.390/0.552 | 0.038/0.077/0.093 | 0.076/0.461/0.665 | **0.071/0.242/0.334** |\n\n*Note: Three values in each cell correspond to: high-frequency energy ratio/RMS gradient/RMS curvature*\n\n- **Smoothness Advantage Analysis:** Our method demonstrates advantages in trajectory smoothness due to the progressive generation strategy from low-frequency to high-frequency:\n  - **Low-frequency prior constraints:** In initial stages, the model generates low-frequency information containing main motion patterns and overall trends, naturally possessing smooth characteristics\n  - **Noise filtering mechanism:** Through hierarchical processing, early low-frequency reconstruction effectively filters high-frequency noise interference, providing a more stable foundation for subsequent iterations\n  - **Progressive refinement:** Each iteration refines based on the smooth foundation of the previous level, avoiding sudden changes and discontinuities that might be introduced by direct full-frequency generation\n\n**3.Sample Efficiency:** As shown in Table 4 of the main paper, we achieve significantly better performance than other methods using only 10 demonstration samples.\n\n**4.Noise Robustness:** We conducted two types of noise experiments: 1. Low-quality demonstration experiments using expert demonstrations generated with lowered reward thresholds (5 tasks used, hammer task cannot generate low-reward expert demonstrations) 2. Gaussian noise experiments adding different intensity Gaussian noise to high-quality demonstrations (6 tasks used with 0.025, 0.05, 0.1 std)\n\n-**Noise Robustness Results (Success Rate %):**\n\n### 1. High-Quality Reference\n| Method | Pick Out of Hole | Soccer | Stick Pull | Hammer | Door | Pen | Average |\n|:------:|:----------------:|:------:|:----------:|:------:|:----:|:---:|:-------:|\n| **DP3** | 12 | 22 | 57 | 100 | 53 | 50 | 49 |\n| **Mamba** | 25 | 28 | 55 | 100 | 59 | 55 | 54 |\n| **Ours** | 30 | 32 | 62 | 100 | 65 | 59 | 58 |\n\n### 2. Low-Quality Demonstration Experiment\n| Method | Pick Out of Hole | Soccer | Stick Pull | Door | Pen | Average |\n|:------:|:----------------:|:------:|:----------:|:----:|:---:|:-------:|\n| **DP3** | 10 | 15 | 41 | 47 | 33 | 29 |\n| **Mamba** | 16 | 20 | 40 | 44 | 34 | 31 |\n| **Ours** | 20 | 17 | 43 | 49 | 32 | 32 |\n\n### 3. Gaussian Noise Experiment\n\n**Noise Intensity 0.025 std**\n| Method | Pick Out of Hole | Soccer | Stick Pull | Hammer | Door | Pen | Average |\n|:------:|:----------------:|:------:|:----------:|:------:|:----:|:---:|:-------:|\n| **DP3** | 26 | 25 | 63 | 0 | 55 | 53 | 37 |\n| **Mamba** | 18 | 33 | 61 | 0 | 50 | 57 | 37 |\n| **Ours** | 32 | 33 | 60 | 70 | 52 | 56 | 51 |\n\n**Noise Intensity 0.05 std**\n| Method | Pick Out of Hole | Soccer | Stick Pull | Hammer | Door | Pen | Average |\n|:------:|:----------------:|:------:|:----------:|:------:|:----:|:---:|:-------:|\n| **DP3** | 21 | 19 | 51 | 0 | 27 | 55 | 29 |\n| **Mamba** | 23 | 25 | 60 | 0 | 51 | 57 | 36 |\n| **Ours** | 25 | 30 | 60 | 51 | 46 | 55 | 45 |\n\n**Noise Intensity 0.1 std**\n| Method | Pick Out of Hole | Soccer | Stick Pull | Hammer | Door | Pen | Average |\n|:------:|:----------------:|:------:|:----------:|:------:|:----:|:---:|:-------:|\n| **DP3** | 7 | 18 | 50 | 0 | 24 | 38 | 23 |\n| **Mamba** | 3 | 14 | 39 | 0 | 21 | 39 | 19 |\n| **Ours** | 20 | 27 | 52 | 15 | 32 | 49 | 33 |\n\n**Experimental Results Analysis:**\n\nWe discovered the following important phenomena:\n\n  - **Low-quality demonstration impact:** For low-quality demonstrations, all methods show significant performance decline, validating the importance of high-quality training data.\n\n  - **Moderate noise promoting generalization:** Adding lower-intensity Gaussian noise may actually promote generalization ability in 20 random environment tests for certain tasks. For example, in the pen spinning task, Gaussian noise may unintentionally increase hand movement variation, adding action diversity and leading to higher success rates than without noise. Similarly, the Pick Out of Hole task also shows better test performance when low-intensity noise is added.\n\n  - **Hammer task noise sensitivity analysis:** Our method performs particularly well on hammer tasks because hammer tasks require high precision and are particularly sensitive to noise. Our low-frequency to high-frequency progressive generation mechanism has natural anti-noise advantages:\n    - **Low-frequency anti-noise characteristics:** Low-frequency components inherently have stronger noise resistance, establishing a stable action foundation in initial stages\n    - **Progressive filtering:** Each frequency domain level's reconstruction process acts as natural noise filtering\n    - **Structured constraints:** The coarse-to-fine generation process ensures that key action structures are preserved even under noise interference"}, {"review_id": "lNXx70bp4e", "reviewer": "Reviewer_FE3t", "summary": "The paper proposes a new method, FreqPolicy, aiming to learn motions progressively with respect to different frequency. FreqPolicy uses a conditioned diffusion model to iteratively generate motions at different levels. The authors compare the method on several simulators to prove its efficiency.", "questions": "1.\tI think the major problem is the lack of comparison with FAST. FAST also use DCT and decompose the action sequence in the frequency domain. Both the motivation and the method part is very similar. The authors claim that FAST leads to “significant information loss” (L70). It would be definitely helpful if the authors can compare the FAST as a baseline on success rates, reconstruction loss on evaluation set, etc.\n\n2.\tFor the robomimic benchmark (L166), can you explain the composition of 10 degrees of freedom for robot arm with parallel grippers?\n\n3.\tFor figure 3(b), what is the x-axis (frequency domain scale)? For frequency domain scale = $1.00$, why the success rates for most of the tasks are still not close to 1.0? How lossy is the reconstruction?\n\n4.\tHow to represent $y_k$ in the discrete space?\n\n5.\tFor the experiment results Figure 4, it seems that more iteration (>=4) will lead to a fluctuated success rate (4 > 6 > 7 > 5, etc). If would be better if the author can shed more lights on this. For instance, showing the reconstruction loss of different iterations, etc., will be helpful.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- I really like the visualization and analysis of section 3.2. The motivation of the paper is clear, and is not a pure combination of different methods and tricks, which is insightful.\n\nWeaknesses:\n\n- The writing of the method section is little confusing. It seems that some information in figure 2 is not presenting in the paper, and it took me a while to understand the whole pipeline.\n\n- Comparing with FAST [26] as a baseline seems to be necessary since the similarity of these two methods (both operate in the frequency domain, both use DCT, both try to decompose the actions, but FAST is discrete while FreqPolicy is continuous).\n\nIn general, I like this paper, and I'm willing to raise the score if the authors can solve the problems.", "was_revised": true, "final_justification": "My major concerns are (1) lack comparison with FAST; and (2) definition problem related with DOF. Now the issues have been resolved and I would like to raise the score.", "rebuttal": "Thank you for your detailed questions and constructive suggestions. Your feedback from multiple perspectives - method clarity, experimental comparisons, and technical details - has helped us improve the quality of our paper. We sincerely appreciate your thorough review!\n\n**W1.The writing of the method section is little confusing. It seems that some information in figure 2 is not presenting in the paper, and it took me a while to understand the whole pipeline.**\n\nThank you for the feedback. We have verified that all content and symbols in Figure 2 are mentioned in the method section. To further eliminate confusion, we provide additional clarification of the overall pipeline:\n\n**Core Mechanisms:**\n- **Diffusion Decoding Mechanism:** As shown in Figure 2, FreqPolicy aims to generate actions containing full-frequency information from action tokens at any given frequency level (controlled by index tokens).\n\n- **Specific Training Process:** At index=k, the observation features and corresponding k-level reconstructed actions $y^k$ are input to FreqPolicy's MAE to obtain latent representation $z^k$. $z^k$ serves as the condition for the diffusion model, guiding the generation of full-frequency actions from noise sampling.\n\n- **Hierarchical Frequency Autoregressive Iteration:** The generation process is structured as a for-loop, strictly traversing frequencies from low to high. In the k-th iteration(assuming k iterations divide into k frequency bands):\n  - **Low-frequency extraction:** extracts k-1 level low-frequency actions $y^{(k-1)}$ from the previous prediction's full-frequency action $\\hat{x}^{(k-1)}$ through IDCT\n  - **Conditional encoding:** inputs $y^{(k-1)}$ and robot observations O as conditions to FreqPolicy encoder/decoder, obtaining latent embedding $z^{(k-1)}$\n  - **Diffusion generation:** uses $z^{(k-1)}$ as condition input to the diffusion model, generating complete actions $\\hat{x}^k$ containing all frequency details\n\n- **Progressive Generation:** At each stage, the model revisits and optimizes the complete action sequence based on increasingly frequency-level and refined actions, enabling progressive generation from coarse-grained to fine-grained.\n\nConcrete examples can be found in our response to Reviewer Zaqb's weakness 1. We will further optimize the clarity of expression in the method section in the revised version.\n\n**Q1.I think the major problem is the lack of comparison with FAST. FAST also use DCT and decompose the action sequence in the frequency domain. Both the motivation and the method part is very similar. The authors claim that FAST leads to \"significant information loss\" (L70). It would be definitely helpful if the authors can compare the FAST as a baseline on success rates, reconstruction loss on evaluation set, etc.**\n\nYour observation about the missing FAST comparison is very insightful. Following suggestions from both you and Reviewer wtRU, we conducted detailed comparative experiments.\n\n- **Method Route Clarification:** We and FAST adopt two different frequency domain processing approaches:\n  - **FAST Route:** Uses quantization to filter high-frequency components, introducing discretization losses.\n  - **Our Route:** Different tasks need different frequency levels - high-DOF tasks need full-frequency information. We use continuous tokens to preserve action space properties.\n\n- **Experimental Setup:** We tested both FAST+Diff and FAST+AR baseline methods.\n\n- **Experimental Results Comparison (Success %):**\n\n    | Method | Hammer | Door | Pen | Pick Out of Hole | Soccer | Stick Pull | Avg |\n    |:------:|:------:|:----:|:---:|:----------------:|:------:|:----------:|:---:|\n    | FAST+Diff | 13 | 18 | 16 | 32 | 26 | 50 | 26 |\n    | FAST+AR | 0 | 0 | 0 | 0 | 0 | 0 | 0 |\n    | Ours | 100 | 65 | 59 | 30 | 32 | 62 | 58 |\n\n- **FAST Reconstruction Error Analysis (RMSE):**\n\n    | Task | Hammer | Door | Pen | Pick Out of Hole | Soccer | Stick Pull |\n    |:----:|:------:|:----:|:---:|:---------:|:------:|:----------:|\n    | FAST_RMSE | 0.59 | 0.57 | 0.63 | 0.26 | 0.23 | 0.32 |\n\n- **Key Findings:**\n    - **FAST+AR Failure Reason:** Due to frequent token length mismatch issues in FAST's decoder, resulting in all-zero outputs and experimental failure. FAST authors acknowledge this problem in their official repository and suggest using π_0 diffusion decoding as a solution.\n\n    - **Information Loss Verification:** RMSE data clearly validates our claim in Line 70 about \"FAST leading to significant information loss.\" Reconstruction errors in high-DOF tasks (Hammer: 0.59, Door: 0.57, Pen: 0.63) are significantly higher than low-DOF tasks (Pick Out of Hole: 0.26, Soccer: 0.23, Stick pull: 0.32).This explains why FAST achieves reasonable performance on low-DOF tasks but shows degraded success rates on high-DOF tasks.\n\n    - **Performance Difference:** Our method's average success rate (58%) significantly outperforms FAST+Diff (26%), with particularly evident advantages in high-DOF tasks.\n\n    - **Fundamental Method Distinction:** We use full-frequency information with completely lossless reconstruction (guaranteed by DCT's reversible property), while FAST's filtering and quantization operations indeed introduce performance losses.\n\n**Q2.For the robomimic benchmark (L166), can you explain the composition of 10 degrees of freedom for robot arm with parallel grippers?**\n\nThe original robomimic dataset consists of eef_pos + eef_rot + gripper = 3+3+1 = 7 DOF. In DP tasks, eef_rot uses 6D rotation representation, resulting in 3+6+1=10 DOF.\n\n**Q3.For figure 3(b), what is the x-axis (frequency domain scale)? For frequency domain scale = 1, why the success rates for most of the tasks are still not close to 1.0? How lossy is the reconstruction?**\n\n- **X-axis Definition:** Frequency domain scale represents the range of frequency domain used (0-100%), i.e., the ratio of k/T. For trajectories with shape B×T×dim, DCT transformation maintains B×T×dim. Using k-level frequency information means adopting B×k×dim.\n\n- **Frequency Information Distribution:** Although trajectory information is mainly concentrated in low frequencies, high-frequency information becomes important as task complexity increases, leading to success rate differences across different frequency domain ranges (see frequency domain analysis videos in supplementary materials).\n\n- **Success Rate Explanation:** When frequency domain scale = 1.0, success rates do not reach 1.0 because:\n   - Testing follows DP/DP3 benchmark protocol, evaluating across 20 random environments\n   - Even RL algorithms that obtain expert demonstrations cannot guarantee 100% success rates across 20 test environments\n   - This reflects task success criteria rather than single-task reconstruction loss\n\n- **Reconstruction Loss:** When using 100% frequency domain, reconstruction loss is necessarily 0, guaranteed by DCT's reversible property. Our method considers all frequency domain information, while FAST's filtering of high-frequency components leads to reconstruction loss.\n\n**Q4.How to represent in the discrete space?**\n\nThe representation of $y^k$ in discrete space is achieved through Discrete Cosine Transform (DCT/IDCT):\n\n- **Representation Process:**\n    - **1.Original Trajectory:** Shape B×T×dim\n    - **2.DCT Transformation:** Convert to frequency domain, maintaining shape B×T×dim  \n    - **3.Frequency Selection:** Select the first k frequency components, obtaining B×k×dim\n    - **4.Zero Padding:** Set remaining (T-k) high-frequency components to 0, reconstructing B×T×dim\n    - **5.IDCT Reconstruction:** Obtain action space $y^k$ corresponding to k-level frequency domain through inverse DCT\n\n- **Discrete Property Clarification:** The \"discrete\" here refers to the discrete nature of DCT operations - transformation of finite-length sequences, not numerical quantization. The values in $y^k$ remain continuous floating-point numbers, actually representing k-level reconstruction of the original action sequence, preserving information from the first k frequency components.\n\nThis representation method enables us to precisely control information granularity levels in the frequency domain space.\n\n**Q5.For the experiment results Figure 4, it seems that more iteration (>=4) will lead to a fluctuated success rate (4 > 6 > 7 > 5, etc). If would be better if the author can shed more lights on this. For instance, showing the reconstruction loss of different iterations, etc., will be helpful.**\n\n- **Success Rate Fluctuation Reasons:** When iter>4, the following issues exist:\n\n    - **Over-fine High-frequency Division:** High-frequency bands are excessively subdivided, falling into long-tail distribution problems where the model struggles to learn overly subtle high-frequency detail variations\n    - **Noise Interference:** Over-fine subdivision causes the model to learn noise in high-frequency signals, affecting performance\n\n- **Reconstruction Loss Explanation:** Regardless of the number of iterations, when the final frequency level reaches full-frequency domain T level, the reconstruction loss of DCT transformation is always 0. For example:\n    - **iter=1:** Frequency domain divided as 0-100% (full frequency domain)\n    - **iter=2:** Equally spaced division into 50% and 100%, where the 2nd iteration uses 0-50% frequency domain information as condition to generate full-frequency domain actions\n\n    This explains why our method achieves lossless reconstruction while maintaining the flexibility to balance efficiency and precision through iteration control."}, {"review_id": "UoKCbInmKg", "reviewer": "Reviewer_Zaqb", "summary": "The paper focuses on an important, often overlooked, aspect of learning robotic policies: *how to represent actions*. Mainly, transforming the actions to the frequency domain provides a certain structure that is effective in identifying global motions (low frequency) and local, fine, motions (high frequency). The proposed action representation and method, FreqPolicy, is composed of hierarchical frequency components (progressively learned), implemented as an autoregressive-diffusion model, and is evaluated on multiple benchmarks resulting in better performance.", "questions": "Questions:\n* I wrote my questions under “Weaknesses” and “Minor”.\n\nSuggestions:\n* I wrote my suggestions under “Weaknesses” and “Minor”.\n\n\nI'm willing to increase my score given a sufficient response to my concerns (mainly, the clarity issue) and upon reading the other reviews.", "limitations": "Very sparse limitations section in the appendix. I can think of some limitations that arise from the complexity involved in synching all the components together (masking and diffusion). I also recommend adding a word or two on this in the main text (e.g., in the Conclusions section).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n* Well-motivated: the analysis performed on robotic actions and motions justifies the proposed approach.\n* Extensive benchmark.\n* Faster inference than previous diffusion-based policies.\n* Some generalization capabilities (though the overall performance is not great, i.e. <30).\n* Real-world demonstration.\n* Very detailed appendix.\n* A promise for an open-source code.\n\n**Weaknesses**:\n* I find the clarity of the method lacking and the entirety of Sections 3.4-3.5 hard to follow. I’m confused regarding the stages and the notations. I would suggest adding a proper example to clarify the sampling/generation pipeline. E.g., “Consider a 3-dimensional action space and a trajectory of 5 timesteps. We apply DCT for $k=...$ levels… For this case, $x_i$ is… $y_i$ is … For hierarchical generation, consider $l_i \\in {...}$ and we perform 4 iterations resulting in…”. See “Minor” for more.\n* The reported performance improvement seems marginal compared to DP3/Mamba Policy in Tables 1-3\n* Misleading results: in Section 4.2 - “Inference Efficiency” - the authors mention that the method is faster compared to diffusion policies, but in Figure 4, to outperform DP3, the method requires 3-4 iterations, which has longer inference time than DP3. I would like a clarification for that.\n* The method includes several algorithmic components (e.g., masking, diffusion,...) which might make it a bit complex to implement and tune (though the authors report the hyper-parameters on the appendix).\n* No ablations in the main text (only in the appendix). I recommend writing something about it in the main text and short conclusions/insights.\n* Limitations: very sparse limitations section in the appendix. I can think of some limitations that arise from the complexity involved in synching all the components together (masking and diffusion). I also recommend adding a word or two on this in the main text (e.g., in the Conclusions section).\n\n**Minor**:\n* Introduction - line 44 - you introduce the term “energy” but do not explain its meaning. I also recommend making the caption of Figure 1 a bit more detailed and guide the reader what to look for in the figure (as explained in the main text). In general, this term is used extensively throughout this text and it should be explained in the context of this work or there should be a reference to where it is explained at the very least.\n* Figure 2: “We” -> “It”/”The model” (or change all the verbs, i.e, “transforms”, “learns”, “reconstructs”). Also, maybe add numbers to the steps in the figure so the reader can understand where the pipeline starts.\n* Please add variable dimensions to section 3.3 (e.g., $x \\in \\mathbb{R}^{a \\times b}$). Also, according to the notation prior to Equation 1, shouldn’t $n$ run from 1 to $N$? There is no $x_0$ in your notation.\n* Section 3.5: I’m a bit confused regarding the notations for the index. What is the relationship between $k$ and $l_i$?\n* Line 298: “For all simulation experiments, we use 4 iterations, whereas for real-world experiments, only 1 iteration is used”. Iterations of what? Diffusion? Which variable is that referring to?\n* Table 4: I assume the numbers are success rates? Please explicitly mention thai in the caption/table.", "was_revised": true, "final_justification": "My concerns were mostly about the readability and the clarity of the paper. Overall, I like the idea of the paper and I think there is a contribution here to the community, even though the reported results are mostly marginal in compared to other methods (but the method seems to be faster).", "rebuttal": "Thank you for your valuable feedback on method clarity, and notation standards. We will implement improvements in the revised version.\n\n**W1. I find the clarity of the method lacking and Sections 3.4-3.5 hard to follow. I'm confused regarding the stages and the notations. I would suggest adding a proper example to clarify the sampling/generation pipeline.**\n\nHere is a detailed example:\n\n- **Detailed Example:** Consider a 3-dimensional action space and a trajectory of 16 timesteps: $x = [x_0, x_1, ..., x_{15}]$ with shape(1×16×3), using 4 iterations with frequency domain indices uniformly divided as index = [0, 4, 8, 12].\n\n- **Training Process:**\n    1. **Random Sampling:** Uniformly sample frequency level k between 0-16, assume we get k = 8\n    2. **DCT Transformation:** $x$ (1×16×3) → $X = [X\\_0, X\\_1, ..., X\\_{15}]$(1×16×3), where $X\\_0$ is the lowest frequency component and $X\\_{15}$ is the highest frequency component\n    3. **Select First k Components:** Since k=8, select first 8 components $[X_0, X_1, ..., X_7, 0, 0, ..., 0]$ (1×16×3)\n    4. **IDCT Reconstruction:** Obtain 8-level reconstruction $y^8$ (1×16×3)\n    5. **MAE Encoding:** Input ($y^8$ + Obs features + index embedding(k=8)) $\\rightarrow$ Output latent $z^8$\n    6. **Diffusion Loss:** Use $z^8$ as condition to compute diffusion loss\n\n- **Sampling Process: (Index embedding omitted in here for brevity)**\n    1. **Iteration 1 (index=0):** Obs features + initial $y^0$ = all zeros(when k=0, reconstructed $y^0$ is also all zeros, perfectly corresponding to the state without any frequency domain information) → MAE encoding → $z^0$ → Diffusion prediction $x^1$ (1st iteration) → DCT, take first 4 components, IDCT reconstruction\n\n    2. **Iteration 2 (index=4):** Obs features + 4-level reconstruction $y^4$ → MAE encoding → $z^4$ → Diffusion prediction $x^2$ (2nd iteration) → Take first 8 components, IDCT reconstruction\n\n    3. **Iteration 3 (index=8):** Obs features + 8-level reconstruction $y^8$ → MAE encoding → $z^8$ → Diffusion prediction $x^3$ (3rd iteration) → Take first 12 components, IDCT reconstruction\n\n    4. **Iteration 4 (index=12):** Obs features + 12-level reconstruction $y^{12}$ → MAE encoding → $z^{12}$ → Diffusion prediction $x^4$ (4th iteration, final full-frequency output)\n\n    This entire process achieves progressive generation from low-frequency to high-frequency components.\n\n**W2. The performance improvement seems marginal compared to DP3/Mamba Policy in Tables 1-3**\n\nWe believe our method demonstrates significant advantages across multiple important dimensions:\n\n- **High-DOF Task Advantages:** We achieve significantly higher success rates on high-degree-of-freedom dexterous manipulation tasks. As future demands for high-DOF dexterous operations grow, the advantages of our modeling approach will become more apparent. For example, in Adroit and DexArt benchmarks, our method shows significant advantages on complex manipulation tasks.\n\n- **Flexible Efficiency-Precision Trade-off:** We can efficiently complete tasks under different inference speed requirements by adjusting iteration counts. This adaptability is not available in other methods.\n\n- **Multi-dimensional Performance Improvements:** Beyond success rates, we have significant advantages in: trajectory smoothness, sample efficiency, and noise robustness.\n\n- **Future Potential:** Exploring the combination of autoregressive and diffusion approaches in the action domain has important significance, providing new insights for understanding and modeling action space from different perspectives.\n\nDetailed results can be found in our response to Reviewer wtRU Question 3.\n\n**W3. Misleading results: in Section 4.2 - \"Inference Efficiency\" - the authors mention that the method is faster compared to diffusion policies, but in Figure 4, to outperform DP3, the method requires 3-4 iterations, which has longer inference time than DP3. I would like a clarification for that.**\n\nRegarding inference efficiency comparison:\n\n- **Comparison with Diffusion Policy:** Our method is significantly faster because Diffusion Policy uses 100 sampling steps, while reducing sampling steps leads to performance degradation.\n\n- **Comparison with DP3:** DP3 uses 10 sampling steps. When we use 3 iterations: (1) inference speed is comparable, and (2) performance is superior to DP3.\n\n- **Experimental Setup:** To ensure fair comparison, we directly use the original paper's values and test according to the same benchmark.\n\n**W4. The method includes several components (masking, diffusion) which might make it complex to implement and tune.**\n\nWe made special efforts to simplify the design:\n\n- **Hyperparameter Reuse:** Training hyperparameters such as learning rate directly follow DP3 settings\n- **Diffusion Model Reuse:** Directly use MAR's diffusion model architecture  \n- **Standardized Masking Strategy:** Adopt MAR's standard masking configuration, based on open-source MAE model\n- **Lightweight Model:** Only 63M parameters (vs DP3's 255M, see Appendix A), memory consumption ~4.5G, trainable on 2080TI\n- **Open Source Support:** Will open source complete implementation and process documentation\n\n**W5. No ablations in the main text (only in the appendix). I recommend writing something about it in the main text and short conclusions/insights.**\n\nDue to space constraints, we present key ablation study results through Figure 4 in the main text. We will add a brief summary of ablation experiments and main insights in the main text in the revised version.\n\n**W6. Limitations: sparse limitations section in the appendix. Some limitations that arise from the complexity involved in synching all the components together (masking and diffusion). I also recommend adding a word or two on this in the main text (e.g., in the Conclusions section).**\n\nRegarding system complexity, we have addressed this in W4. We will improve the revised version by: Describe the inherent complexity in the main text conclusion.\n\n**M1. Introduction - line 44 - you introduce the term \"energy\" but do not explain its meaning. I also recommend making the caption of Figure 1 a bit more detailed and guide the reader what to look for in the figure (as explained in the main text). In general, this term is used extensively throughout this text and it should be explained in the context of this work or there should be a reference to where it is explained at the very least.**\n\nRegarding the \"energy\" concept in Figure 1:\n\n- **Definition:** Energy represents the proportion of signal energy preserved within a specific frequency domain range, calculated as:\n- $E_{total} = \\sum |X(k)|^2$ (k = 0 to N-1)\n- $E_p = \\sum |X(k)|^2$ (k = 0 to [(N-1)×p%]), where p = 10, 30, 60, 100\n- Energy = $E_p/E_{total}$\n\n- **Role:** As shown in Figure 1, energy proportion reflects the contribution of different frequency domain ranges to signal reconstruction quality. Low-frequency band (0-10%) only preserves 59% energy with high RMSE, while extending to mid-frequency band (0-30%) preserves 95% energy with significantly reduced reconstruction error. This observation provides important insight: adopting the strategy of using the previous frequency domain level as condition for the next level, achieving progressive generation from low-frequency to high-frequency.\n\nWe will clearly define the \"energy\" concept and add complete mathematical formulas in the revised version.\n\n**M2. Figure 2: \"We\" -> \"It\"/\"The model\" (or change all the verbs). Also, maybe add numbers to the steps in the figure so the reader can understand where the pipeline starts.**\n\nWe will improve the revised version by:\n- Modify verb forms in Figure 2 to maintain consistency\n- Add numerical markers to process steps in the main text to improve readability\n\n**M3. Please add variable dimensions to section 3.3 (e.g., ). Also, according to the notation prior to Equation 1, shouldn't run from 1 to ? There is no in your notation.**\n\nThank you for the correction: $x = [x_1, ..., x_N]$ should be $x = [x_0, ..., x_{N-1}]$. We will add variable dimension information in Section 3.3 in the revised version.\n\n**M4. Section 3.5: I'm a bit confused regarding the notations for the index. What is the relationship between k and l_i?**\n\nThe relationship between k and $l_i$ is as follows:\n- $l_i$: represents the frequency domain level index corresponding to the i-th iteration\n- In sampling process: $l_i$ is determined according to iteration number i, where $l_i$ is the specific value of k\n- In training process: k is randomly sampled between 0 and N\n\nIn short, $l_i$ is the specific instantiated value of k during sampling, while k is the random variable during training. We will further clarify the definitions and relationships of these symbols in the revised version.\n\n**M5. Line 298: \"For all simulation experiments, we use 4 iterations, whereas for real-world experiments, only 1 iteration is used\". Iterations of what? Diffusion? Which variable is that referring to?**\n\nThe \"iterations\" here refer to the number of iterations in multi-frequency level inference, i.e., the number of intermediate levels in the process of generating actions layer by layer from 0-level to N-level frequency domain, not diffusion steps.\n\nSpecifically:\n- **Simulation experiments:** Use 4 iterations, achieving precise action generation through layer-by-layer refinement\n- **Real-world experiments:** Use 1 iteration, directly generating full-frequency domain actions. Since real-world dynamic handover tasks need to respond to rapid environmental changes, single iteration can meet fast inference requirements. Although multiple iterations can provide more fine-grained control, single iteration is more suitable for the high real-time requirements of such dynamic tasks.\n\n**M6. Table 4: I assume the numbers are success rates? Please explicitly mention that in the caption/table.**\n\nYes, the values are success rates. We will explicitly label \"success rate\" in the table caption in the revised version."}]

Author rebuttal:

Meta-review: The paper proposes FreqPolicy, a frequency-domain autoregressive diffusion policy that learns robot actions from low to high frequencies, and the reviewers agree it is a fresh way to obtain smoother and faster visuomotor control across 2D and 3D benchmarks. Before rebuttal the reviewers were mildly positive, praising the originality and broad experiments but noting hard-to-follow method description, missing baselines, and only modest improvement over strong time-domain policies. After rebuttal the authors gave new FAST comparisons, clearer pipeline examples, and extra real-robot results; the reviewers say these answers solve their concerns and raised ratings to unanimously borderline accept or accept, though limited empirical gains remain a concern. The AC therefore recommends acceptance as a poster.
Decision: Accept (poster)