Real-Time Robot Execution with Masked Action Chunking
Authors: Haoxuan Wang
Gengyu Zhang
Yan Yan
Yuzhang Shang
Ramana Rao Kompella
Gaowen Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Robot Manipulation
Real-time Execution
Primary area: applications to robotics, autonomy, planning

Abstract: Real-time execution is essential for cyber-physical systems such as robots. These systems operate in dynamic real-world environments where even small delays can undermine responsiveness and compromise performance. Asynchronous inference has recently emerged as a system-level paradigm for real-time robot manipulation, enabling the next action chunk to be predicted while the current one is being executed. While this approach achieves real-time responsiveness, naive integration often results in execution failure. 
Previous methods attributed this failure to inter-chunk discontinuity and developed test-time algorithms to smooth chunk boundaries. In contrast, we identify another critical yet overlooked factor: intra-chunk inconsistency, where the robot’s executed action chunk partially misaligns with its current perception. To address this, we propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking, enabling the policy to remain resilient under mismatches between intended actions and actual execution during asynchronous inference. In addition, we introduce a prefix-preserved sampling procedure to reinforce inter-chunk continuity.
Overall, our method delivers more reliable policies without incurring additional latency. Extensive experiments in both simulation and real-world settings demonstrate that our method enables faster task execution, maintains robustness across varying delays, and consistently achieves higher completion rates.

Published as a conference paper at ICLR 2026
REAL-TIME ROBOT EXECUTION WITH MASKED ACTION CHUNKING
Haoxuan Wang 3 Gengyu Zhang'.3, Yan Yan', Yuzhang Shang?. Ramana Rao Kompella', Gaowen Liu Universiity of Illinois Chicago ?University of Central Florida
"Cisco Research
ABSTRACT
Real-time execution is essential for vyber-physical systems such as robots. These systems operate in dynamic real-world environments where even small delays undermine responsiveness and compromise performance. Asynchronous infer- ence has recently emerged ulation, enabling the next action chunk to be predicted while the current one is being executed. While this approach achieves real-time responsiveness, naive in- tegration often results in execution failure. Previous methods attributed this failure to inter-chunk discontinuity and developed test-time algorithms to smooth chunk boundaries. In contrast, we identify another critical yet overlooked factor: intra- chunk inconsistency, where the robot's executed action chunk partially misaligns with its current perception. To address this, we propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking. enabling the policy to remain resilient under mismatches between intended ac- tions and actual execution during asynchronous inference. In addition, we intro- duce a prefx--peeerred sampling procedure to reinforce inter-chunk continuity. Overall, our method delivers more reliable policies without incurring additional latency. Extensive experiments in both simulation and real-world settings demon- strate that our method enables faster task execution, maintains robustness across varying delays, and consistently achieves higher completion rates. Video demos are available at this project page
system-level paradigm for real-time robot manip-
INTRODUCTION
The deployment of large-scale models has revolutionized AI capabilities across various do- mains (Liu et al. 2023 Esser et al., 2024 Zheng et al. 2024 OpenAl 2025). In robotics, Vision- Language-Acun (VLA) models (Wu et al. 2023 Brohan et al. 2023b a Kim et al., 2024; Black eet al., 2024 Intelligence et al. 2025a Yang al. 2025 Zhong et al. 2025) have emerged
promising direction for translating human instructions and sensory observations into physical ac- tions. While real-time responsiveness is desirable across all such applications, its importance varies by context. For language or image models, slower responses primarily lead to increased waiting time but won't result in generation failure. In contrast, for cyber-physical systems like robots op- crating in dynamic, real-world environments, the absence of real-time responsiveness can mean the difference between successfully filling cup and catastrophically spilling juice onto the table.
A real-time robot control system must ensure continuous stream of executable actions so that the robot never runs idle (Hester et al., 201T). A straightforward approach is reduce the time cost of action pequisition-primmnne nication-such that it falls below the robot's control period. Although a variety of model-level techniques have been proposed to accelerate model prediction (Leviathan al. 2023 Bolya et al.) 2023 WW et al. 2024; Lin et al. [2024] [Shang et al. [2024] Xu et al. | [2024] Shukor et al. 2025). these methods typically sacrifice accuracy for speed. Moreover, model sizes continue to scale, control frequencies increase, and deployment conditions vary in real-world settings, such approaches be- come increasingly limited in their ability to guarantee real-time performance.
arising from VI.A action generation and network commu-
Action chunking (Zhao et al. 2023 Chi et al. 2024), where a policy predicts and executes a se- quence of actions per inference step, offers partial solution to real-time control. Although it has

---

Published as a conference paper at ICLR 2026
achieved state-of-the-ar results in dexterous manipulation, action chunking inherently reduces sys- tem reactivity to sudden state changes (Liu et al., 2025) and introduces discontinuities at chunk boundaries (Black et al. 2025). These limitations have motivated the search for more general system-level inference frameworks, among which asynchronous inference (Shukor et al., 2025) has recently emerged
promising solution for real-time robotic execution. Unlike synchronous inference (Chi et al., (2024). which pauses until future actions are available, asynchronous inference predicts upcoming actions while executing the current ones, ensuring a continuous supply of actions. However, incorporating asynchronous inference into VLA-based control systems is nontrivial: when coupled with action chunking, it amplifies existing limitations and results in substantial performance
degradation (Liu et al. 2025 Black et al. 2025 Shukor et al.
2025)
Previous methods (Zhao et al. 2023 Liu et al. 2025 Black al. 2025) addressing this challenge primarily aimed to mitigate inter-chunk discontinuity, seeking a balance between long-term co- herence and short-term reactivity. These approaches typically treat the already executed actions as informative priors and apply test-time refinements to the upcoming action chunks. However, such refinements are either heuristic in nature and prone to catastrophic failure (Zhao et al., (023), or exploit the generative and inpainting capabilities of diffusion//lowmatchhe 2023) while incurring additional latency (Liu et al.] 2025; Black et al.] 2025). Moreover, these works overlook critical failure mode: intra-chunk inconsistency, arising from misalignment be- tween observations and the actions executed within a single chunk.
models (Song et al.,
In this work, aim to improve both inter-chunk continuity and intra-chunk consistency when inte- grating asynchronous inference with chunking policies. We first formulate intra-chunk inconsistency as a partial mismatch between observations and executed actions, leading shift in intra-chunk distribution between training and sampling. To address this, we propose Real-time Execution with Masked Action Chunking (REMAC) to learn corrective adjustments to the pretrained policy by masking arbitrary portions of action chunks. In parallel, we enhance inter-chunk continuity by re- fining the sampling pipeline to incorporate previously executed actions as informative priors.
Our method introduces no additional inference delay compared to the pretrained policy (Black et al., 2024 2025), and can be seamlessly integrated into existing VLA frameworks. Extensive experi- ments across 12 simulated tasks and three real-world settings demonstrate its effectiveness, show- ing higher success rates, faster task completion, and smoother robot dynamics under varying delay conditions. Moreover, approach can be combined with existing test- time algorithms, further underscoring its utility to produce stronger backbone policies for asynchronous execution.
RELATED WORKS
2.1 VLA AND ACTION CHUNKING
An emerging line of robotics research focuses on developing generalist policies capable of perform-
ing broad spectrum of tasks and transfer across different environments and robot embodiments.
Vision-Language-Actiin (VLA) models (Brohan et al. 2023b a Wu et al.] 2023 Kim et al.) 2024
Black et al. 2024 Bjorck al., 2025 Intelligence et al. 2025a Shukor al. 2025) have become
leading approach in this pursuit. By conditioning on high- inputs such as video frames, natural language instructions, and proprioceptive signals, VLAs generate the corresponding low-level motor
control commands. Among these models, diffusion and flow-matching approaches (Lipman et al.)
2023
Chi et al., (024) have gained prominence for their ability to generate high-quality actions efficiently, and they serve the representative setting for our work.
Moreover, scaling to long-horizon manipulation requires temporal abstraction: models must predict not just the next command, but coherent segments of behavior. Inspired by principles of human mo- tor control (Lai et al. 2022). recent systems structure behavior into temporally extended sequences, action chunks, which are generated by the VLA and executed by low-level controller. This action-chume chuuonnchuninng paradigm enables visuomotor policies to operate over meaningful sequences and provides a foundation for alternative strategies of action execution.
2

---

Published as a conference paper at ICLR 2026
Robot
Ex
Auce
P=3, HPZ, d=1
VLA
Observationi))
VLA
Pred(t-h)
Policy Outputs
Observation
Intra-chunk
A
Pred[t]
Pred[()
Inconsistency
Executed
Executefti
Actions
Executelt]
Inter-chunk Discontiuuity
(a) Synchronous Inference
(b) Asynchronous Inference
(c) Inference Challenges
Figure I: Illustration of execution paradigms. Arrowed lines of the same style indicate processes occurring simultaneously. (a) Synchronous inference: VLA prediction and robot execution alter- nate sequentially. (b) Asynchronous inference: VLA prediction runs concurrently with execution. (c) Although asynchronous inference enables real-time execution, it introduces two performance- degrading challenges: exacerbated inter-chunk discontinuity and intra-chunk inconsistency.
2.2 EXECUTION STRATEGIES FOR CHUNKED POLICIES
While action chunks temporally consistent within single segment, discontinuities and distri- bution shifts often arise at chunk boundaries. To mitigate this issue, ZZhao et al. (2023) proposed Temporal Ensembling (TE), which aggregates the overlapping portions of consecutive chunks to improve smoothness. Liu et al. (2025) highlighted the importance of balancing long-term consis- tency with short-term reactivity, introducing Bidirectional Decoding (BID), a method that samples multiple candidate predictions and selects the optimal one. More recently, RTC (Black et al.) 2025) explored real-time execution strategies under asynchronous inference by framing the task as a test- time inpainting (Pokle et al. 2024) problem, proposing to leverage the prior chunk to warm-start planning for the next and applics gradient-buse based corrections to the predicted chunk. However, RTC introduces additional computational latency, which may degrade performance. In this paper, likewise target real-time execution under asynchronous inference, but instead adopt a training- adaptation approach. Our method introduces additional inference overhead, consistently outper- forms existing strategies, and can be seamlessly combined with test-time correction methods.
3 PRELIMINARIES
Let v, v (A.lo:) denote the action chunking policy, where A, = [ag. "+41....,a(+P.... represents the predicted action chunk of length P. o, denotes the current observation, and indexes the con- troller timestep. The parameter P is the prediction horizon. During rollout, only the first actions executed, where h is the execution horizon with 1 < h < P. In real-world robotic manipula- tion, latency arises from stages such as VLA action prediction and network communication, which together define the inference delay: continuous quantity representing the time lag between the acquisition of o and the availability of the corresponding A, in the control queue. Following Black al. (2025), we discretize this delay as d 6/ At where is the continuous inference latency and At is the high-level controller sampling period. For simplicity, our definition of inference delay excludes observation latency and sub-timestep- delays.
Flow-matching policies. We follow prior work (Liu et al. 2025 Black et al.) 2025 and consider action-chunking policies trained via flow matching (Lipman et al. 2023). In standard flow matching, the model learns velocity field that maps intermediate action states toward expert trajectories. Training proceeds by minimizing an & objective between the predicted flow á and the ground-truth target At inference time, flow-matching policy generates action chunk by first sampling an initial latent action sequence A, from a Gaussian prior. The final chunk is then obtained by integrating this sample along the learned velocity field v. over normalized time variable [0, 1]. Using n integration steps, the update rule is:
A = A; + 1 o,, T),
(1)
where or is the current observation. After iterating over [0. 1], the final state A is used the predicted action chunk.
Synchronous inference is the conventional paradigm adopted in many works (Zhao et al. 2023 Black et al. 2024, Bjorck et al., (025). where the robot executes all actions within the current execution horizon before supplying the latest observation to the VLA to infer the next chunk (Fig-
3

---

Published as a conference paper at ICLR 2026
ure Ta)). For synchronous inference to achieve real-time execution, the condition ò < At (i.e., 0) must hold, which is practically unattainable. For example, with a 50 Hz control frequency (At 20 ms) and Ta (Black et al., (024) as the VLA model, action generation alone requires 76 ms on an NVIDIA RTX 4090 GPU, with additional overhead from preprocessing, disk I/O, and network transmission further increasing latency. As control frequencies rise for fine-grained tasks and larger VLA models are deployed, the effect of inference delay only becomes more pronounced. Conse- quently, synchronous inference fundamentally constrained, resulting in jerky transitions between chunks and extended execution times that undermine policy effectiveness.
Asynchronous inference. in contrast, achieves real-time execution by ensuring that actions are al- ways available (F)). While this produces smoother trajectories, it introduces two challenges Inter-chunk discontinuity. Let A and A2 denote two consec- utive action chunks sampled from trajectories Ti and To with execution horizon h. If Ti and To coincide up to timestep t but diverge thereafter, inter-chunk discontinuity arises at the boundary, yielding incoherent transitions between chunks. Intuitively, because action chunk represents only a local segment of a trajectory, A and A may originate from different latent expert modes, Al- though both Ti and Tz are valid continuations of the same past, switching between these modes at the chunk boundary can introduce a large jump in the action sequence, producing jerky or out-of- distribution motion during execution. @ Intra-chunk inconsistency. Assume the policy v (A,)or) perfectly captures the underlying environment dynamics and therefore yields the optimal action se- quence. Given perception og. the optimal executed chunk should fully correspond to Aç. However, under inference delay d with execution horizon h, the first d actions executed are instead taken from A,-h- This results in intra-chunk inconsistency, where these inherited prefix actions become suboptimal for the current state, because they were conditioned on O:-A rather than Og, creating a perception action mismatch within the chunk. Figure [e) illustrates an example with 3, h = 2. and d 1, highlighting both challenges. Note that inter-chunk discontinuity also arises under synchronous inference, but is further exacerbated in asynchronous settings.
that degrades performance:
METHODOLOGY
We consider a flow-mat policy V. (o.) Our objective is to mitigate both intra-chunk inconsistency and inter-chunk discontinuity by learning delay-aware policy v (AAAoor d), built upon (A,))). By conditioning on the inference delay d, the new policy learns predict action chunks that are reliable despite the uncertainty introduced by delayed execution. The following sections describe the components of our approach in detail.
4.1 MASKED ACTION CHUNKING
Given the pretrained policy v, (A, lo:), asynchronous inference executes the first few actions from the previous chunk A, while the remaining actions are drawn from A,, where h denotes the exe- cution horizon. This misalignment between perception and execution induces a train-test mismatch: supervision on the unexecuted early actions during pretraining can provide misleading signals, lead- ing to off-distribution priors. To address this issue, we introduce REMAC, which incorporates the following learning strategies. The details are described below and summarized in Alg. [
Prefix Masking. We begin by shifting the learning emphasis toward the to-be-executed action chunk segment, only focusing on the portion of the trajectory that directly influences execution while leaving the unexecuted prior intact. Formally, for the predicted flow ú E RxDD from V. (Ar| jo,, d) and the ground-truth target restricts supervision to the executable portion of each chunk:
IR PxD, we introduce a delay- cendyicond prefix mask my that
ma = {m))==) = 1(r 2 d].
(2)
where indexes the timesteps within the chunk, 1[:] is the indicator function, and d ~ U40,.... ... P- }} is a uniformly sampled random inference delay. By incorporating the prefix mask into the con- ventional flow-matching loss, we obtain the following objective:
P-1 m -ü, u-
Σr=o
C.
(3)
1, 1 P-1 mz
max
4

---

Published as a conference paper at ICLR 2026
This formulation directly builds on the masking strategy, restricting supervision to the unmasked segment while excluding the already-commitee prefix. By randomly sampling across all valid in- ference delays, the policy is exposed to the full spectrum of conditions fromm the trivial unmasked case (d = 0) to extreme masking scenarios (d h). This mitigates intra-chunk inconsistency by strengthening the policy's adaptability to uncertainty in unexecuted actions, yielding rollouts that remain resilient even under imperfectly predictable environment dynamics. Moreover, the strategy enhances the policy's predictive capacity and equips single model to handle all valid delay settings, removing the need to train separate policies for different delays.
Self-conditioned Curriculum. The actions executed within the inference delay during rollout can serve as informative test-time priors for refining subsequent action chunks. However, during training the executed portion of the chunk is unavailable, unless the current policy is rolled out from scratch for each input sample-a a strategy that is prohibitively expensive and impractical, even in simulation. To address this limitation, we introduce self-conditioned curriculum scheduling method that leverages the pretrained policy to imitate test-time conditions, thereby improving both intra-chunk consistency and inter-chunk discontinuity.
As a concrete example, consider standard flow matching, where the training input is constructed by linearly interpolating between Gaussian noise and the ground-truth action chunk A,, Instead of directly using the ground-truth chunk, we modify the formulation to incorporate self-conditioning from the pretrained policy, defined as:
Å, A + sg((1 )),
? B [0, 1].
(4)
Here, As denotes the action chunk predicted by the pretrained policy, sg(-) (-) is the stop-get -raddrede operation, avoiding unnecessary gradient updates. The mixing parameter scheduled based on the training progress, annealing from = 1 (pure ground-truth input) to = 0 (pure self-conditioned input). The resulting mixture A, is then used in place of A₁ for interpolation.
Therefore, the model input evolves during training: in the early stages, ground-truth actions serve as stable anchors, while in later stages the model is gradually condi- tioned on its own predictions yet still re- quired to produce accurate outputs. This scheduled curriculum not only stabilizes training, but also mitigates exposure bias by aligning training inputs with test-time conditions. In doing so, it enables the model to learn corrective adjustments over the pretrained policy, refining its own pro- posals and improving robustness to distri- bution shifts and compounding rollout er- rors. Figure 7 illustrates several schedul-
Algorithm 1 REMAC
Input: Delay set Sa, dataloader D, epochs E Initialize: Pretrained policy v-).), target policy V-(-) 1: for e e 0 to E - do
2: Sample d Sy and compute md via Eq.
3:
for all (o., U2) D do
4:
A, Integrate(v..(0))
5:
A, Interpolate A,, uz)
Eq.
ú, < (=
LoRA enabled
7:
i, f- v (o,,AA)
> LoRA disabled
8:
Update v. by minimizing Eq.
6
9:
end for
10:
end for
ing functions for a, among which we adopt the piecewise linear decay as the default.
Residual Alignment. In addition to standard supervision against the ground truth, we introduce a A-matching term on the unmasked action chunk. This term explicitly aligns the induced correction with the residual between the pretrained policy's 'sicc'sspia prediction and the ground-truth target. Specifically, let ū denote the flow estimate from the pretrained backbone v, "A((Ao,,) The correction then encouraged to match the residual toward the target through the following objective:
[ - ū7) - -0mmmmmmmmm m7 1+)
LA
(5)
max 1, [ m3
While mathematically related to Eq. 3 the two objectives emphasize different aspects: Eq. Bq.Bua en- forces direct alignment with the ground truth, whereas Eq. ejxpliicyy models the residual adjust- ment relative to the pretrained policy. Empirically, we observe that incorporating LA into training yields substantial performance improvements.
5

---

Published as a conference paper at ICLR 2026
Finally, the overall training objective is defined
C Amam + Aaha.
where Am 0.01 and A 0.01 the weighting coefficients.
4.2 PREFIX-PRESERVED SAMPLING
Having obtained the new policy V (A,|or, d), we further adjust the sampling pipeline to align with the training procedure and enhance inter-chunk continuity. Firstly, the initial action state A is no longer drawn from the Gaussian prior. Instead, it is initialized using the can be executed actions during the delayed time steps, denoted as A, PxDD, whose first / - h entries are filled with the last P-h actions from the previous predicted chunk, while the remaining entries are zero- eero-initialized.
Secondly, during the sampling process, the overlapping segment between the currently executing chunk and the upcoming chunk are preserved as the executed ones, and only the remaining portion of the action chunk is newly synthesized. Specifically, if the sampling process integrates the learned velocity field v over n integration steps, let m E {0, } denote the delay-conditioned prefix mask and A, is the intermediate action state at integration time т, then:
1 A, = m - (A} + - PA o,, 7)) + (1 m O AP,
where @ denotes wlement-wise multiplication. A special case arises for the first action chunk gener- ation at rollout initialization, where no previous actions have yet been executed. this case, we set AP 0 random Gaussian noise to indicate the absence of prior actions, and apply the standard integration rule in Eq.
4.3 IMPLEMENTATION DETAILS
Model Adaptation. We adapt the pretrained policy using the parameter-efficient finetuning tech- nique LoRA etal model. The reasons we adopt LoRA instead of full model finetuning are twofolds. First, we con- ceptualize the transformation from vr (A, o₂) to v.r (A, or, d) a distributional adjustment with respect to the pretrained policy, rather than a wholesale re-learning of the policy under differ- objective. Second, LoRA offers a flexible mechanism that can leave the original parameters untouched, thereby preserving the predictive capacity of the pretrained model and making it well suited to our method. During implementation, the LoRA module can be further extended with an additional projection matrix that incorporates the prefix mask to enrich the input representation.
202T), al.
introducing at most 1.5% additional parameters relative to the original
Training and Sampling. During training, control the prefix prefix-mask mask coverage by uniformly sam- pling d in Eq. 2 from a gradually shrinking interval iq, {max]. where Qmax is fixed and is linearly annealed from amax down to final Jmin- Importantly, Qmax and Qmin nor limits on the inference delays that the model can handle; rather, they are mask-induced perturbations during training. The total loss defined in Eq. 6 then used update the LoRA parameters. During action sampling, compared to the conventional approaches in prior works (Chi et al.) 2024 BBlack et al. 2024). the model additionally takes the pre-determined esti- mated inference delay and the previous action chunk as inputs for the next action chunk prediction. The LoRA modules can be further merged into the backbone model layers, such that no additional time cost is introduced.
that determine the strength of
Deployment Framework. For simulation benchmarks, we follow the setting of Black et al. (025), in which the predicted actions are deliberately segmented and concatenated according to predefined delays and execution horizons. For the real-world deployment, we adopt setup similar to Shukor et al. (2025). in which actions are predicted on a remote server and transmitted to the robot via gRPC. The robot maintains a queue of actions to execute while sending observations to the server at the same frequency as its control loop. The remote server, hosting the VLA model, likewise maintains queue of incoming observations and performs action prediction frequently as possible using the most recent inputs and states. The inference delay is estimated on the robot client side the maximum of the most recent few measured delays.

---

Published as a conference paper at ICLR 2026
car launch
cartpole thrust
calapull
catober v3
Average Solve Rate
00
Ours
chain lander
grosp easy
h17 unicycle
hard lonar londer
0G
RTC
BID
Average Execution Time
mijo half cheetah
mnjo swimmer
inje walker
trempoline
Inference Delay
Inference Delay
Figure 2: Performance comparison in Kinetix environments. Left: Solve rates for individual tasks under varying inference delays. Right (top): Average performance across all environments. Our method consistently outperforms baselines under all delay settings and exhibits smaller performance degradation as delay increases. Right (bottom): Average execution time across all environments. Our method requires fewer steps and achieves faster task completion.
EXPERIMENTS
In this section, present experiments evaluating the effectiieneess of our method against bascline approaches under various inference delay settings. We also perform ablation studies on different components to assess their individual contributions. Our empirical analysis spans both simulation and real-world tasks, providing a comprehensive evaluation of the proposed method.
5.1 KINETIX ENVIRONMENT
We first evaluate our method on the Kinetix simulator (Matthews al. 2025). following the protocol of Black et al. (2025). The benchmark comprises 12 highly dynamic and stochastic environments specifically designed to test asynchronous execution. Expert policies are first trained under a bi- nary success reward using RPO (Rahman & Xue 2022), from which training data are collected via demonstration generation. Flow-matching policies are then learned through imitation learning (Zare et al. 2023 Argall et al. 2009). For each task, the trained policies are configured with prediction horizon of 8. The inference delay d ranges from 0 to 4, and the execution horizon for each delay is chosen within max (1, d) to 9 - d, ensuring continuous action availability without gaps between consecutive chunks. LoRA is applied to all linear layers except the time-embedding and AdaI.N layers (Peebles & Xie, (023). with each LoRA layer assigned rank of 4. For each fixed delay value, evaluation metrics are reported as averages over all corresponding execution horizon settings.
Baselines. The baselines we compare with include: (1) Naive Asyne, which directly uses the pre- trained policy and executes actions from the most recently generated action chunk. (2) Bidirectional Decoding (BID) (Liu et al. 2025). a test-time method that samples multiple candidate predictions and applies rejection sampling select the optimal one, aiming to balance long-term consistency with short-term reactivity. (3) RTC (Black et al. 2023). also a test-time execution strategy that leverages an inpainting algorithm (Pokle al.] 2024). It applies gradient-based corrections to the predicted actions, using the executed actions during the inference delay as priors. We omit the base- line of Temporal Ensembling (TE) (Zhao al.] (203). both our experiments and Black et al. (2025) show that TE substantially underperforms the other baselines in this simulation benchmark, even falling behind Naive Asyne.
Figure 2 reports the per-task and average performance across 12 tasks, measured by both task success rate and completion time. Generally, we observe a consistent decline in performance as inference delay increases across all methods. This degradation stems from the growing mismatch be- tween actions and observations, which amplifies intra-chunk inconsistency, and from the increasing influence of the previous chunk, which exacerbates inter-chunk discontinuity. Our method consis-
Results.

---

Published as a conference paper at ICLR 2026
Table 1: Effectiveness of the different components in REMAC.
Method
d=0
d=1
d=2
d=3
d=1
Naive
0.828
0.702
0.639
0.525
0.451
5000
LoRA
0.825
0.710
0.630
0.510
0.428
Prefix masking
0,863
0.825
0.732
0.729
0.636
Self-conditioned curriculum
0,848
0.837
0.805
0.762
0.710
Inference Delty'
CA (Ours)
0,888
0.879
0.859
0.817
0.779
Figure 3: Schedule comparison.
tently outperforms all baselines across all delay settings, with especially pronounced gains under larger inference delays. It also achieves shorter average execution time than competing methods, demonstrating the ability to produce more efficient policies for faster task completion.
Our method also improves performance under d = O. We attribute this to the masked ac- tion chunking formulation, which encourages the policy to enforce stronger coherence and temporal dependencies even without inference delay, thereby enhancing its predictive ability. Moreover, as the inference delay increases, our method exhibits both a smaller performance drop and lower rise in execution time compared to baselines. This trend highlights the robustness of our approach under increasing delays, underscor- ing its potential a versatile solution for asynchronous inference.
Table 2: Integration with test-time methods.
Method
d-0
d-1
d-2
d-3
d-4
Ours
0.888
0.879
0.859
0.817
0.779
BID
0.888
0.880
0.862
0.821
0.781
RTC
0.888
0.879
0.864
0.826
0.791
Ablations and Analysis. We conduct ablation studies to evaluate the contribution of each compo- nent and further analyze the extensibility of our approach.
Table [ reports the contribution of each component in our method. First, adding LoRA alone-wiind without modifying the training paradigm-yielde no performance gain, indicating that the effectiveness of our approach cannot be attributed merely to the increase in parameters. In contrast, progressively incorporating the components described in Sec. Sec. leads to consistent im- provements, with the full method achieving the highest overall success rate. Detailed per-task results are provided in Sec. [
Figure compares variations of the self-conditioned curriculum schedule introduced in Eq. 4 We compare with two baselines: one that initializes exclusively with ground-truth actions (a 1) and another that relies entirely on self-conditioned inputs (a 0). The results show that the curriculum schedule improves both performance and training stability relative these extremes. Training with pure ground-trute inputs suffers from exposure bias, since during policy rollout the sampling process relies on the model's own predictions. Conversely, training with fully self- conditioned inputs often destabilizes early learning. The curriculum schedule mitigates both is- sues by gradually transitioning from ground-truth self-conditioned inputs.
Our method can further be integrated with other test- time approaches such as BID and RTC, since modest,
it only modifies the backbone policy. Mableeishove that, although the improvements integration consistently provides additional performance gains across delay settings, with larger improvements observed under higher delays. This demonstrates both the compatibility of our approach with existing test-time strategies and its potential as plug-and-play method.
We also ablate the choice of (max and Gmin (Sec. 4.3 to assess how mask coverage affects perfor- mance. As shown Fig. Toja), performance changes only marginally across configurations, indi- cating that REMAC is not highly sensitive to these hyper-parameters. Larger values particularly a larger 9mio produce slightly worse results. We use 9max = and qmin = 0 in practice.
We further show that REMAC is not limited to flow-matching policies. In Sec. E.5 we integrate REMAC into the Transformer-based ACT (Zhao al. 2023) framework, where it consistently outperforms both the naive asynchronous baseline and the LoRA-only baseline. These results demonstrate that REMAC readily extends beyond flow matching and can be seamlessly incorpo- rated into diverse action-chunking architectures.
5.2 REAL-WORLDD ENVIRONMENT
Setup. We employ a Franka Research 3 robot (7-DoF arm) (Haddadin 2024)
equipped with The additional
parallel-jaw grippers and adopt the DROID setup (Khazatsky al. 2025) (Figure 6
8

---

Published as a conference paper at ICLR 2026
Table 3: Average completion progress. Progress measured by discrete scores corresponding to the sub-tasks completed.
Spereoo Salve Rate
0.8
0.7
Method
Grasp-Easy
Grasp-Mediuen
Grasp-Hard
Synchronous
0,805
0.718
0.670
0.6
RTC
0.5
Naive (Shukor et al. 2025
0.825
0.825
0.460
Malve
Sync
TE (Zaa et
2023
0.825
0.868
0.717
75mma Iejected Delay
+150ma
RTC Black 2025
0.823
0.848
0.758
Figure 4: Performance under delay injections on Grasp-Hard.
Ours
0.903
0.943
0.812
tactile sensors of the grippers are disabled and repurposed to narrow the gripping range, thereby acting as constraints for evaluating fine-grat tee-graiedd control. This design ensures that the tasks demand precise action execution, making them well suited for assessing performance under asynchronous inference. For the backbone VLA, we use To (Black et al. 2022), configured with a prediction hori- zon of P 50, and adopt its memory-efficient variant for finetuning. A total of 200 trajectories are collected for model adaptation, with LoRA A layers of rank 8 inserted only into the action expert mod- ule. After finetuning, the LoRA weights are merged into the backbone, ensuring that no additional computational overhead is introduced at inference.
During exccution, the robot operates control frequency of 15Hz, corresponding to At 67ms. The execution horizon is fixed at h 8, and sampling is performed over 10 steps. Processing through the VLA model without any test-time strategies takes approximately 76-80ms per obser- vation. Since the model is hosted on a separate server, communication over LAN introduces an additional network delay of 34-40ms, while data processing and disk writing contribute a further 10-20ms. In total, the end-to-end inference delay is roughly 122-140ms, corresponding to effee- tive inference delay of d = 2 or 3.
Task Design and Measurement. We evaluate our approach three single-arm grasp-and and and-place tasks (Grasp-Easy, Medium, Hard) of varying difficulty. The tasks range from manipulating simple objects, such as a eucumber, to more challenging ones, such Rubik's cube whose sides are only lem shorter than the gripper's jaw gap, requiring precise control. Placement targets are either a plate or bowl, with the latter posing greater difficulty and demanding finer manipulation accuracy.
Task performance is measured using a stage-based solve rate that evaluates progress through four steps: (1) reaching the object, (2) gripping and lifting it, (3) moving it toward the target location, and (4) placing it correctly into the container. For each task, we conduct 30 evaluation trials per method, with each trial capped at 300 steps, amounting to a total of 6 hours of robot execution time. The initial position and orientation of the objects are randomized across trials.
Baselines. We compare our method against four baselines: (1) Synchronous inference, the widely used baseline in prior works (Black et al. 2024 Kim et al. 2024 Hu et al. 2025 Kim et al. 2025 Pertsch et al. 2025), which executes an entire predicted action chunk and then pauses until new actions received. (2) Naive Asyne, and sampling is run as frequently as possible such that the most recent actions are queued. (3) Temporal Ensembling (Zhao et al.] 2023). an extension of Naive Asyne that aggregates overlapping actions across consecutive chunks by weighted averaging. (4) RTC, the state-of-the-art baseline but it introduces an additional 55- - 644m of inference latency. We omit BID from comparison, as is substantially more time- and computation computation-inteniive intensive than the other methods (Black et al. 2025). and is therefore not a competitive baseline in real-world settings.
Results. Table B reports the average completion progress for each task, showing that our method achieves higher completion rates across all tasks. During execution, synchronous inference produces frequent pauses, often leading to unintended object drops and inaccurate localization. Asynchronous baselines generate smoother trajectories without pronounced jerkiness. However, Naive Asyne and Temporal Ensembling remain prone to premature or delayed grasping and placement. In contrast, RTC suffers from the additional inference delay it introduces.
Figure reports results with additional latency injections of 75ms and 150ms, simulating deploy- ment under slower hardware and network conditions. Even with total inference delays of 3 - 5, our method consistently outperforms all baselines, demonstrating robustness to varying delay levels. Interestingly, Naive Asyne performs comparatively better under larger delays, while RTC exhibits

---

Published as a conference paper at ICLR 2026
Sync
Start
Finish
RTC
Finish
Ours
Start
Finish
Velocity
Acceleration
Kinematic
redis
Durs
RBC
Figure 5: Visual comparison of different methods under 150ms injected delay. Top: Task com- pletion progress, where our method achieves faster completion. Bottom: Average robot kinematics during task execution on Grasp-Hard. Our method exhibits smoother trajectories and faster task completion speed.
significantly degraded performance. We attribute this to the fact that larger delays correspond to longer execution horizons in real-world settings: less frequent chunk switching reduces inter-chunk discontinuity, but the test-time adjustments made by RTC can have adverse effects.
Figure Sl compares task completion progress and corresponding robot kinematics. We evaluate under an injected 150ms delay and adopt cases where all policies achieve successful rollouts, ensuring fair comparison while amplifying differences across methods. Qualitatively, our method completes tasks within shorter time. Quantitatively, analysis of average robot velocity and acceleration 15 trials shows that synchronous inference produces abrupt, periodic kinematic changes, whereas asynchronous inference methods yield smoother trajectories. Among these, method achieves the most stable dynamics with fewer abrupt changes, highlighting both speed and stability.
CONCLUSION
In this paper, we address the problem of effective real-time robot manipulation under asynchronous inference. We identify two critical challenges in this setting exacerraaeeeed inter-chunk discontinu- ity and intra-chunk inconsistency-ann approaches, REMAC learns corrective adjustments on top of a pretrained policy through a masked action chunking strategy and preiix-preserve sampling pipeline, while introducing no additional inference delay. Extensive experiments in both simulation and real-world benchmarks demonstrate that our method is robust varying delay conditions and achieves faster task completion.
propose REMAC to mitigate them. Unlike prior test-time
ETHICS STATEMENT
This work focuses on algorithmic development and evaluation for asynchronous inference in robotic control. All experiments were conducted either in simulation or with a physical Franka Research robotic arm controlled laboratory setting. Our method is intended solely for research purposes and does not present foreseeable risks of harmful deployment. We believe our work fully adheres to the ethical standards and guidelines of the community.
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we provide detailed algorithmic description in Alg. and implemen- tation choices in See. 4.3 In the experimental section, clearly specify the datasets, training protocols, evaluation metrics, and implementation details, including inference settings. In addition, source code is released.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This research is supported by NSF IIS-2525840, CNS-2432534, ECCS-2514574, NIH IRFIMH113364-01 and Cisco Research unrestricted gift. This article solely reflects opinions and conclusions of authors and not funding agencies. We gratefully acknowledge the individuals who provided valuable guidance and feedback for this project, including Changda Tian (FORTH) for hardware-related suggestions and Hongkai Zheng (Caltech) for insightful advice inpainting.
REFERENCES
Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5):469-48, 2009. ISSN 0921-8890. doi: http://oo..a/////ooo//..oooo...0
URL https: /wwp: sciencedirect.
Com/ oct.e/nc/arrtaaee//////012111181
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Wang, Jiannan Xiang. Yuqi Xie, Yinzhen Xu, Zhen- jia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu, Gr0Ot nl: An open foundation model for generalist humanoid robots, 2025. URL https:://axivy wtg. alos / 2203. 11.5503..7734
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. No: A vision- language-action flow model for general robot control, 2024. URL htt.tps://arxi..oott abs 2410.2410222
Kevin Black, Manuel Y. Galliker, and Sergey Levine. Real-time execution of action chunking flow policies, 2025, URL http/t:s/////iiii/ /ttp:iiw.rg//bb//55...o.. 0733333366679999
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster, 2023. URL. httts:://rrxi>>> wwtpaiiwwwooorooaooxop/oa/a
2210. 09461
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Hen- ryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt- 2: Vision-languagg-aatton models transfer web knowledge to robotic control, 2023a. URI. htttps://arxiv orgooogggo. abs 22077 15818
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Kuang- Lee, Sergey Levine, Yao Lu, Utsav Malla, Deek- sha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vin- cent Vanhoucke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu. Tianhe Yu, and Brianna Zitkovich. Rt-1: Robotics transformer for real-world control at scale, 2023b. URL https://erxifs www. a a0// 06817

---

Published as a conference paper at ICLR 2026
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024. URL
https://arxiv: org/ ptwp/bs/////aooooo////g/////aa 2303. 04137
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL https://axxivorr///a 1403.03200
Sami Haddadin. The franka emika robot: A standard platform in robotics research. IEEE Robotics & Automation Magazine, 3144:136-144, 2024. doi: 10.1 10.1109//RA220444445517
Todd Hester, Michael Quinlan, and Peter Stone. A real-time modelbbasseddiiinffrremme reinforcement learning architecture for robot control, 2011. http:tttsa//ar/x...r////////////
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https:
wlaaaii..ggaaaaaae.aa. 09688
Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations, 2025.
http:/bs/22411 14803
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenherg. Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. To.5: vision-languagg-actioo model with open-world generalization, 2025a. URL https: //arxiv.. org/abs/ 250aab//2555.. 16054
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair. Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling. Haohuan Wang, Lili Yu, and Ury Zhilinsky. To.s: Assion-language-action model with open- spen-world generalization, 2025b, URL https: /rrxiv. hrg/abs/2504. 16054
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Jocy Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, Patrick Tree Miller, Jimmy Wu, Suneel Belkhale, Shivin Dass, Huy Ha. Arhan Jain, Abraham Lee, Young- woon Lee, Marius Memmel, Sungjae Park, Ilija Radosavovic, Kaiyuan Wang, Albert Zhan, Kevin Black, Cheng Chi, Kyle Beltran Hatch, Shan Lin, Jingpei Lu, Jean Mercat, Abdul Rehman, Pannag R Sanketi, Archit Sharma, Cody Simpson, Quan Vuong, Homer Rich Walke, Blake Wulfe, Ted Xiao, Jonathan Heewon Yang, Arefeh Yavary, Tony Z. Zhao, Christopher Agia, Ro- han Baijal, Mateo Guaman Castro, Daphne Chen, Qiuyu Chen, Trinity Chung, Jaimyn Drake, Ethan Paul Foster, Jensen Gao, Vitor Guizilini, David Antonio Herrera, Minho Heo, Kyle Hsu, Jiaheng Hu, Muhammad Zubair Irshad, Donovon Jackson, Charlotte Lc. Yunshuang Li, Kevin Lin, Roy Lin, Zehan Ma, Abhiram Maddukuri, Suvir Mirchandani, Daniel Morton, Tony Nguyen, Abigail O' O'Neill, Rosario Scalise, Derick Seale, Victor Son, Stephen Tian, Emi Tran, An- drew E. Wang. Yilin Wu, Annie Xie, Jingyun Yang. Patrick Yin. Yunchu Zhang, Osbert Bastani, Glen Berseth, Jeannette Bohg. Ken Goldberg, Abhinav Gupta, Abhishek Gupta, Dinesh Jayara- man, Joseph Lim, Jitendra Malik, Roberto Martín-Martin, Subramanian Ramamoorthy, Dorsa Sadigh, Shuran Song, Jiajun Wu, Michael C. Yip. Yuke Zhu, Thomas Kollar, Sergey Levine, and Chelsea Finn. Droid: A large-scale in- ti-the-wild robot manipulation dataset, 2025. URL https:///rxiiv org. /200. 12945
12

---

Published as a conference paper at ICLR 2026
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong. Thomas Kollar, Ben- jamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model, 2024. URL httpst//aaxiii..orr abs /2ep/2466..o 09246
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning visasiiiiiinnlnnnggggntttatiii models: Opti- mizing speed and success, 2025. A.KLLttto///arr...o////////2222
Lucy Lai, Ann Zixiang Huang, and Samuel Gershman. Action chunking as policy compression. PsyArXiv, 2022.
Yaniv Leviathan, Matan Kalman, and Yossi Matias, inferencee from transformers via speculative decoding, 2023. URL
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-awarc weight quantization for Ilm compression and acceleration, 2024. /Rt.ttpsss//asssv..rr///////////
2306 00978
Yaron Lipman, Ricky T. Q. Chen, Heli Ben Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, 2023. httt:httt////rrt/
org/orgg/aa hbc///b///// /tlp//22100 02747
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL
https: //arxvvv.vvv////a org/ arg/abs//// 2304. 08485
Yuejiang Liu, Jubayer Ibn Hamid, Annie Xie, Yoonho Lee, Maximilian Du, and Chelsea Finn. Bidirectional decoding: Improving action chunking via guided test-time sampling, 2025. URL
https://arxiv. wwtp/aa////o..// 17355
Michael Matthews, Michael Beukman, Chris Lu, and Jakob Foerster. Kinetix: Investigating the training of general agents through open-ended physics-based control tasks, 2025, URL https:
//Ar2iv.orrg/ab//221022222
OpenAl. Chatgpt.
com/ 2025. Large language model (GPT-5).
William Peebles and Saining Xic. Scalable diffusion models with transformers, 2023. URL https:
//Arsiiv.rrr/abs//222.0001 /
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models, 2025. /tt.mtpps.//axx...cc///////////a 0974
Ashwini Pokle, Matthew J. Muckley, Ricky T. Q. Chen, and Brian Karrer. Training free linear image inverses via flows, 2024. URL https: //tpiii////aiiii///a org/ abs 2bw.//1.........
Md Masudur Rahman and Yexiang Xue. polic optiiizztioooiii in deep reinforeement learning, 2022.
Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. urXiv preprint arXiv:2403. 15388, 2024.
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi. Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. Smolvla: A visioon-lagguagg--ettin model for affordable and efficient robotics, 2025. URL /ett:ss//aariiiooorr///////55555 01844
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guidee diffusion models for inverse problems. In International Conference on Learning Reprexentations, 2023.
URL https: http::////p..sssmnnscccssooooo
Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li. Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation, 2023. URL htl.:tt:::/ ptttps://areet //wx//axvvvvvv/ wtt.lv.rrgaabb/s/3.. 13139
13

---

Published as a conference paper at ICLR 2026
Junyi Wu, Haoxuan Wang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Ptq4dit: Post-training quantization for diffusion transformers, 2024. URL https:// htpp/axxiiiii.... www.xii.org/bb//2400000
16005
Bingxin Xu, Yuzhang Shang, Yunhao Ge. Qian Lou, and Yan Yan. freepruner: A training-free approach for large multimodal model acceleration. arXiv preprint arXiv:2411.1544 2024.
Zhenjie Yang. Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision- sasiion-languae-acttioon model in end-to-end au- tonomous driving, 2025.
16278
Maryam Zare, Parham M. Kebria, Abbas Khosravi, and Saeid Nahavandi. A survey of imitation learning: Algorithms, recent developments, and challenges, 2023. URL https://arxiv.
org/ /bgg/ab/// 2309. 02473
Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual ma- abs/
nipulation with low-cost hardware, 2023.
/abbs/3300 2304. 13 705
Zangwei Zheng, Xiangyu Peng. Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412. 2024.
Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang, Shaoyang Guo, Tianrui Guan, Ka Nam Lui, Zhiquan Qi, Yitao Liang, Yuanpei Chen, and Yaodong Yang. A survey vision-language-actiin models: An action tokenization perspective, 2025. URL https: // /arxiv.v.v. org 2007000000
14

---

Published as a conference paper at ICLR 2026
A HARDWARE SETUP
Figure iillssrrate our robot setup, where the gap between the gripper jaws is deliberately reduced to increase task difficulty and require finer control. Although two third-view cameras are mounted, only one third-view camera and the wrist-mounted camera are used for data processing. This config- uration provides both global view of the scene and local, fine-graine perspective of the manip- ulation area, ensuring accurate observation while maintaining a controlled evaluation environment.
Figure 6: Robot setup illustration.
B SCHEDULING FUNCTION VARIANTS
Figure Souureeillltratttte different variants of the scheduling function a. While multiple options are avail- able, we primarily adopt the piecewise linear schedule, as it provides smoother warm-up phase and more stable optimization compared other choices.
Linear
Cosine
Exponential
Piecewise Linear
3.8
344
2001
1986
12:09
110.
2031
43:0
.000
280
2311
1291 Stop
115
Stop
Stop
Stop
Figure Different scheduling functions.
C TASK EXAMPLES
Figure presents wrist-camera views of the tasks included in our experiments. These examples high- light the visual perspectives used for policy input and illustrate the varying levels of manipulation difficulty across tasks.
Grasp-Easy
Grasp-Medium
Grasp-Hard
Figure 8: Examples of our included tasks.
15

---

Published as a conference paper at ICLR 2026
D PER-TASK SIMULATION RESULTS
Figure presents the per-task results corresponding the ablations in Table The detailed metrics demonstrate that each component of our method contributes consistent improvements in success rate across tasks, highlighting the generalizability of the design choices.
Ours
RTC
BID
Naive
cur leanch
cartpolo thout
catspit
cxtcher 4
Car laanch
carppóle thoust
extyplt
cxtcher 43
chats ander
grigp awyy
h17 usigyde
hand loner ander
chale ander
gragp eary
h17 uskoyde
hand hoar ander
zge ball steetab
rge neSSener
Rge wilker
tumpstos
xqc bell ttaetab
mge entrrear
nge vilker
Eurppiin
LoRA
Future- segmmett masking
lacoch
cartpolle
catcher
cartpola thrutt
calohersh
chale loder
uskoyde
hard hoar ander
chan lander
hard laner lander
ball chustab
valke
Exmpeline
raje tall choraa
sge snimazer
walser
Inurpultte
Self-conditioned curriculum
LA
Figure 9: Per-task results for component ablations.
E ADDITIONAL ABLATIONS
E. EFFECT OF ĄMAX AND QMIN
In Figure. a)),), we evaluate the influence of the hyperparameters Jnax and Jmin (Sec. 4.3 by testing multiple combinations under the same training setup. Overall, we observe only minor performance variation across different settings, indicating that REMAC is not sensitive to the exact choice of these values. However, larger values particularly a larger Jmin tend to produce slightly worse performance. This is expected: when Qmin > 0, the prefix of length gmin is always masked during training. which encourages the model to behave over-conservatively during rollout. practice, we adopt Jmas. 4 and Omnn 0. Although this choice happens coincide with the delay range evaluated in our simulation experiments, should not be interpreted as limiting the range of delays REMAC can handle.
E.2 EFFECT LEARNING A MASK EMBEDDING
Our method conditions on the inference delay by converting into prefix mask that is applied during both training and inference. Beyond its role in loss computation and sampling, this mask can also be treated as an additional input signal by projecting into a learnable mask embedding and injecting it into the model. In Figure I)),,, evaluate the effect of introducing such a learned mask embedding under the same training setup. The results show mixed but generally stable outcomes: while few tasks see marginal changes, most tasks retain similar performance. This indicates that REMAC is robust to architectural variations and does not depend critically on whether the delay mask is embedded used directly.
16

---

Published as a conference paper at ICLR 2026
car lounch
cucipole threst
catipec
calchtr v1
0.675
0,850
Solve Rate
thatn badder
group werg
517 entoycla
inndrr
4u: +3. -0
0,800
4m- Baa
Solve
4x-
0,775
nije
trampoline
qu 2,
0.725
"Inference Delay
Inference Delay
(a) Ablation on training mask coverage
(b) Ablation on adding mask embedding
Figure 10: (a) Ablation on the effect of different Jmax and qmin- (b) Ablation on the effect of adding mask embeddings as additional information.
E.3 DATA-EFFICIENCY OF REMAC
To evaluate sensitivity to dataset size, we conducted additional real-world experiments using only 10 demonstrations per task, compared to the total 200 demonstrations used in the main paper. The average completion progress (10 evaluations per task) is shown in Table
below:
Table 4: Effect of Training Data Quantity
#Traj
Grasp-Fasy
Grasp-Medium
Grasp-Hard
200
0.910
0.936 0.905
0.820
30
0.900
0.810
Even with much less data, REMAC maintains performance very close to the 200-trajectory model, with only mild degradation. These results indicate that REMAC is not data-hungry and remains effective in low-data regimes, thanks to its self-supervised masking mechanism that exposes the model to diverse prefix deviations without requiring additional demonstrations.
E.4 EFFECT OF REMAC ON GENERALIZABILITT
We further examine whether REMAC affects the policy's generalization ability. Our fine-tuning pipeline is as follows: we first fine-tune To using our collected demonstrations following the official implementation, updating both the VLM backbone and the action expert. This fine-tuned mo serves as the baseline model for all methods (Naive, RTC, BID, and REMAC). REMAC then applies LoRA only to the baseline model's action expert, while keeping the entire VLM backbone frozen.
We evaluate generalizability under two settings: (i) scene/background variation and (ii) novel lan- guage prompts. For background variation, we remove curtains, replace the table covering, and intro- duce additional distractor objects. REMAC behaves similarly to the baseline model: both policies reliably ground objects seen during fine-tuning despite the visual changes and background noise. However, under unseen language instructions, both the baseline and REMAC fail to perform novel tasks or recognize unseen objects (e.g., "cup," "box"), suggesting that the limitation stems from the underlying fine-tuned policy rather than from REMAC.
We additionally evaluate on #0.5 (Intelligence et al. 2025b), which exhibits stronger open-world generalization under settings. Under unseen language prompts, the fine-tuned mo.5 correctly recognizes unseen objects but still struggles with unseen tasks. Applying REMAC on top of no.5 pre- serves this OOD capability and does not introduce noticeable degradation, confirming that REMAC does not harm the generalization already present in the underlying VLA model.
In summary, REMAC performs only low-rank adjustments the action expert and leaves the grounding and perception capabilities of the VLM untouched. As a result, does not diminish the generalizability of the fine-tuned base model.
17

---

Published as a conference paper at ICLR 2026
E.5 APPLICATION TO OTHER POLICY CLASSES
We further demonstrate that REMAC is restricted to flow-matching policies and can be applied to other policy architectures. While diffusion-style models are natural extension due to their structural similarity to flow matching, additionally show that REMAC is compatible with Transformer- based chunking policies.
To validate this, integrate REMAC into ACT (Zhao et al., 2023) by applying LoRA to its de- coder layers and action head, and replacing the E2 losses used in Eq. and S with ACT's Eι and KL objectives. In Table and we evaluate on two bimanual ACT tasks and report success rate / average return under varying delays. Across all delay settings, REMAC consistently out- performs both the naive asynchronous baseline and the LoRA-only baseline. These results indicate that REMAC generalizes beyond flow-matching models and can be seamlessly incorporated into diverse action-chunking frameworks.
Table 5: Transfer Cube (h = 12)
Table 6: Insert Box (h =h=33 30)
d
Naive
+LoRA
Ours
d
Naive
+LoRA
Ours
4
0.40 / 354.44
0.52 / 335.04
0.74 / 486.78
0
0.14 / 230.74
0.20 / 218.74
0.18 / 245.72
6
0.48 / 348.76 76
0.58 / 352.24
0.72 / 508.28
5
211.77
0.12 179,90
0.18 / 217.12
8
0.46/ / 309.32
0.68 / 422.62
0.68 / 460.86
10
0.14 / 216.98
0.10/ 183.94
0.16/ / 220.68
F ROBUSTNESS TO VARYING DELAY CONDITIONS
Real-world latency can be fluctuating and noisy. We conducted additional evaluations both simu- lation and the real world to assess REMAC's robustness under noisy, rapidly fluctuating, and adver- sarially spiky delay patterns.
(1) Simulation experiments. We fix the true execution delay to d, but deliberately pass incorrect delay values to the policy using the following schemes:
Noisy and rapidly fluctuating delays: For each policy call, we sample the delay from {d
1, d, d + 1} (i.e., 66% inaccurate).
Spiky delays: With 10% probability, we replace the delay with the maximum valid value to simulate infrequent but large latency spikes.
Below report the average performance over 12 Kinetix tasks:
Table Performance Comparison under Different Latency Conditions.
Setting
d=0
d=1
d=2
d=3
d=1
RTC
0.817
0.778
0.733
0.665
0.588
Ours
0.877
0.860
0.832
0.796
0.760
RTC + Noisy & Fluct.
0,816
0.770
0.728
0.653
0.573
Ours Noisy & Fluct.
0.866
0.837
0.799
0.746
0.757
RTC Noisy & Fluct. & Spiky
0.814
0.772
0.728
0.653
0.575
Ours Noisy Fluct. & Spiky
0.820
0.796
0.769
0.718
0.757
Even under extremely ill-conditioned latency sequences, both RTC and REMAC degrade grace- fully- fully-never never catastrophically. Importantly, REMAC under corrupted delays still outperforms RTC under accurate delays, demonstrating strong robustness to delay misestimation.
(2) Real-world experiments. All of our real-world evaluations already operate under realistic network- and cinputte-inuued delay profiles, where delay measurements are noisy and temporally correlated. The measured delay includes multiple sources-VLA inference time, network transmis- sion, file I/O, memory contention, and system-level scheduling jitter. Because the delay measure- ment always includes inference time, it is necessarily historical. Following RTC (Black et al. 2025), the maximum delay observed in a recent window the estimate passed to the policy. Thus, all real-world results (Table and and Figure 4 inherently reflect noisy and imprecise delay estimates, yet REMAC remains robust and performs strongly under these conditions.
18

---

Published as a conference paper at ICLR 2026
To further stress-test the system, artificially соr the delay input by sampling from {d-1, d, d+1, d+2) for every inference step. This induces noisy, fluctuating, and spiky delays. The system exhibits only 1-2 additional failures out of 10 trials, primarily due occasional overes- timation (d+-2), which leads the robot to temporarily pause and exceed the 300-step time limit. This timeout artifact rather than indication of policy instability.
(3) Inherent robustness from discretized delays. bustness property: the inference delay is defined
Asynchronous execution exhibits a natural ro-
8 At
where is the continuous inference latency and At is the controller sampling period (67 ms 15 Hz). For d to fluctuate by 1, the underlying latency must shift by more than 67 ms-an extreme fluctuation rarely observed in practice. This discretization smooths noise in 6, making asynchronous methods, including ours, inherently robust to moderate latency variations.
G LIMITATIONS
Our method is not without limitations. First, it requires specifying a maximum inference delay in advance to ensure that the optimization process covers the full range of possible delays. If the actual delay during execution exceeds this bound, unexpected failure may occur. Second, the approach may demand a substantial amount of finetuning data for masked finetuning, which could limit practicality in settings where data collection is costly or constrained.
H LLM USAGE
We used LLM (ChatGPT) to assist with writing refinement. Specifically, it was employed to im- prove clarity, grammar, and flow of text, as well to adjust tone for academic writing. No content generation, experimental design, or analysis was delegated to the LLM; all technical contributions, mathematical definitions, and experimental results were developed by the authors. The LLM's role was limited to language polishing and presentation, and all outputs were carefully reviewed and edited by the authors.
19


Reviews:
[{"review_id": "bWnB7eryNx", "reviewer": "Reviewer_szGQ", "summary": "This paper addresses real-time robot execution with action chunks. The authors identify _intra-chunk inconsistency_ as a previously overlooked issue. They propose a method for learning a delay-aware policy by masking the portion of the prefix that restricts supervision to only the executable slice of each chunk. The proposed method incurs no additional latency.", "questions": "Please see the weaknesses section above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper makes solid contributions to an important practical problem. Asynchronous inference is a natural system-level solution for real-time execution. Identifying intra-chunk inconsistency as a distinct failure mode adds value.\n- The paper does a thorough experimental analysis, comparing with the state-of-the-art method RTC, and doing ablation studies on each component of the proposed method.\n- Figure 1(c) provides a great concrete example of the problem the paper is aiming to solve.", "weaknesses": "- The paper cites intra-chunk inconsistency as a core motivation for the proposed method, but doesn't provide any direct evidence that this is a major issue.\n- Is there a way to apply this idea to policy classes other than flow-matching?\n- I don't understand the residual alignment term. Don't the two $\\tilde{u}$ terms in eq (4) cancel out, making it equivalent to (2)? Is there a typo somewhere, or am I missing something?\n- The method requires specifying d_max as a hyperparameter during training. Do you have a sense of what happens when the model encounters delays longer than d_max? Does performance degrade gracefully or fail catastrophically?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jglFDXtg2m", "reviewer": "Reviewer_n9Cq", "summary": "This paper studies real-time robot control with chunk-based VLA policies under asynchronous inference, where inference latency causes the robot to execute stale actions while new ones are being generated. The authors highlight a neglected failure mode—intra-chunk inconsistency—in addition to the known inter-chunk discontinuity issue. They propose REMAC, a training-time strategy that applies prefix masking to simulate partial stale-action execution, a self-conditioned curriculum to mitigate exposure bias, and residual LoRA finetuning to correct the base policy. The method increases robustness under varying latencies without adding inference overhead. Experiments on Kinetix and a real Franka arm show improved success rates and smoother execution across delay conditions.", "questions": "1. How robust is REMAC to inaccurate delay estimates or rapidly varying latency?\n2. Does delay-aware finetuning impact generalization to unseen tasks/objects or language inputs?\n3. Would a continuous or learned delay embedding outperform discrete integer conditioning?\n4. How does the method behave when real latency exceeds the maximum trained delay?\n5. Can the approach be extended to handle coupled observation latency, not just action delay?\n6. The method uses a fixed chunk length, but in practice the optimal horizon may vary with latency and task demands. Can the approach be extended to dynamically adjust chunk length based on system delay or task complexity?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- Identifies and formalizes intra-chunk inconsistency as a distinct source of degradation in asynchronous execution.\n- Simple and computationally efficient method that is compatible with existing VLA architectures.\n- Strong empirical evidence across simulation and real-robot settings, including latency sweeps and ablation studies.\n- No additional inference latency, in contrast to recent test-time smoothing approaches (e.g., RTC).\n- Parameter-efficient finetuning strategy that preserves the backbone’s capability.\n- Compatible with BID/RTC test-time methods, demonstrating good composability with existing techniques.", "weaknesses": "- The method assumes access to accurate and bounded delay estimates. It is unclear how robust the approach is when latency measurements are noisy, rapidly fluctuating, or adversarially spiky.\n- Training samples delays uniformly, but real-world latency tends to be bursty and temporally correlated. Additional evaluation under realistic network- and compute-induced delay profiles would strengthen the claims.\n- Finetuning with masked actions may shift the behavior of the underlying VLA model. The paper does not report out-of-distribution or language generalization results, making it difficult to assess whether broader generalization and grounding capabilities are preserved.\n- The method is demonstrated only on flow-matching VLA architectures. Since REMAC integrates with the sampling process and residual flow fields, it is unclear whether the approach directly applies to non-flow controllers (e.g., autoregressive VLA policies or transformer-based continuous action models). A discussion or preliminary evidence on generality across policy parameterizations would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fcom9XWE6S", "reviewer": "Reviewer_NdJX", "summary": "The paper proposes REMAC for asynchronous inference in real-world robotics, handling chunk discontinuities by finetuning learned policies via flow matching. The key is to condition the policy on a ground-truth action prefix during training and use flow matching to predict the remaining actions. By randomly sampling delay d and adjusting the flow matching curriculum, REMAC trains the policy to adapt to varying inference delays. Experiments on both simulation and real-world benchmarks show that REMAC outperforms baselines under high inference delay.", "questions": "1. I'm confused why REMAC outperforms the pretrained policy (Naive) even when d = 0, according to Figure 2. Is it because REMAC uses additional expert data compared with Naive? If the baselines (Naive, RTC, BID) also access the additional expert data, will they outperform REMAC?  \n2. Can we apply the same dataset used in the pretrained policy (i.e., Naive) to REMAC, so that REMAC does not use additional expert data?  \n3. Can you further explain the Residual Alignment loss Eq. 4? What's the difference between Eq. 4 and the Prefix Masking loss Eq. 2?  \n4. In the simualtion tasks, how is the expert dataset D used for REMAC constructed?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. REMAC can adapt to various inference delays d with a single training process, without needing to retrain for each delay. The use of LoRA modules maintains model performance while reducing training overhead.  \n2. Flow matching enables the model to learn fine-grained continuity between action prefixes and optimal future actions, capturing the dependency between earlier and later actions.  \n3. REMAC achieves strong results across both simulated and real-world benchmarks.", "weaknesses": "1. REMAC has structural requirements on the dataset, which needs to contain diverse future action sequences from the same observation. With this requirement, the model can learn to correct deviating action prefixes resulting from inference delay. Without local adjustment samples in the dataset, REMAC may perform no better than behavioral cloning.  \n2. During inference, the agent must know how long inference takes in order to select an appropriate prefix length d from the previous action chunk. In real-world settings with variable communication delays, accurately estimating this delay may be challenging.  \n3. REMAC improves policies' robustness to inference delays only for flow-matching-based policies. If the bottleneck policy is not trained with flow matching, REMAC requires training from scratch.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CFlivIzgi0", "reviewer": "Reviewer_LpbR", "summary": "This paper proposed a method for finetuning a pretrained VLA policy toward residual modifications i.e., adjusting an action chunk from the pretrained policy by taking into account delay due to latency. To do so, the paper introduces a curriculum learning method in which the policy first learns to imitate the expert and then learns to do residual corrections conditioned on a wide range of delay values. For sampling at test-time, the model conditions on the actions being executed from the previous chunk (due to delay) and the sampling procedure enables the model to adjust the trajectory towards more optimal actions.", "questions": "See weaknesses. For me, the most important questions are with regards to clarity and handling latency (see points 1-3 in Weaknesses section).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1. The proposed method has been tested on a wide range of tasks in both simulation and in the real world. The paper offers interesting analysis at a low-level such as the kinematics of the robot under various methods (see figure 5). \n2. The paper compares the proposed method with all the appropriate baselines. \n3. The paper carries out several important ablations and offers interesting insights into how baseline methods can further uplift the performance of REMAC.", "weaknesses": "1. Lack of clarity in the problem formulation, the definition of inter-chunk discontinuity is very unclear (line 153). It is not clear why two trajectories that diverge must not have actions that are different i.e. if the trajectories are similar only up to time $t$ and then the two trajectories diverge to different observations, their action chunks at time $t + h$ should be different? Furthermore, it is not clear what this discontinuity means and what the expected behavior is if there is no discontinuity. Similarly, intra-chunk inconsistency is unclear. While the first $d$ actions are taken from the previous action chunk $\\textnormal{A}_{t-h}$, I do not understand what the perception-action mismatch is. \n2. Lack of clarity in the method discussion. Generally, since there are various choices of implementing flow matching, it would be helpful to either have a preliminaries or an appendix section where you define both the predicted flow and the ground truth flow and, in your notation, specify which policy each is being sampled from. I understand that abstracting these modeling choices helps put the emphasis more on the generality of your method but, at least, some concrete examples will help the reader develop a more concrete understanding of your method. Similarly, the switch to using $\\textnormal{x}_\\mathrm{p}$ to denote the action seems strange. I also think section 4.2 is extremely dense and unclear, from notation to definitions (for e.g., the definition of $f$ in equation 6). \n3. While the method discusses handling temporal consistency, it is unclear whether this method handles the latency issues as discussed in the RTC paper (Black et. al. 2025). It seems like the delay-aware policy would also have high latency, so is it that your method relies on the conditioning on the delay to take care of that? \n\nAs is clear, most of my complaints are with respect to the clarity of the discussion of the problem formulation and the method. I would be happy to raise my score if these issues with clarity are adequately answered. Apart from them, I have some questions about the scalability of the method. \n\n4. The method seems quite data expensive in fine-tuning. For example, for the real world tasks, you collected 200 trajectories. It would be useful to see how the performance changes as we change the size of the dataset. \n5. I am not sure that this method would generalize and I suspect that this might harm the robustness of the pretrained policy. Since you are finetuning offline, it is clear that the REMAC would do well on tasks/environments seen during finetuning – the model knows what the target expert actions are and the model knows what the pretrained policy outputs at these states in the offline dataset. This raises the question of whether or not this fine-tuned model would reliably adapt the pretrained policy’s behavior in unseen states. For example, one issue that might arise is that, at unseen states, the policy might be more uncertain leading to higher entropy of actions from the pretrained policy, and as such your finetuned policy needs to learn how to modify a large number of actions to match the optimal one (which the model does not know yet since this is at test time). Since you are already testing this on $\\pi_0$ which demonstrates some generalization, it would be good to evaluate on some tasks/environments not seen during fine-tuning.\n\n[1] Kevin Black, Manuel Y. Galliker, Sergey Levine. Real-Time Execution of Action Chunking Flow Policies. \n[2] Yuejiang Liu, Jubayer Ibn Hamid, Annie Xie, Yoonho Lee, Maximilian Du, Chelsea Finn. Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)