State-Covering Trajectory Stitching for Diffusion Planners
Authors: Kyowoon Lee
Jaesik Choi
Venue: NeurIPS 2025 poster ()
Keywords: Offline Reinforcement Learning
Trajectory Optimization
Diffusion Models
Trajectory Stitching
Primary area: reinforcement_learning
TL;DR: SCoTS augments offline data by stitching diverse, long-horizon trajectories, enhancing diffusion planners.

Abstract: Diffusion-based generative models are emerging as powerful tools for long-horizon planning in reinforcement learning (RL), particularly with offline datasets. However, their performance is fundamentally limited by the quality and diversity of training data. This often restricts their generalization to tasks outside their training distribution or longer planning horizons. To overcome this challenge, we propose *State-Covering Trajectory Stitching* (SCoTS), a novel reward-free trajectory augmentation method that incrementally stitches together short trajectory segments, systematically generating diverse and extended trajectories. SCoTS first learns a temporal distance-preserving latent representation that captures the underlying temporal structure of the environment, then iteratively stitches trajectory segments guided by directional exploration and novelty to effectively cover and expand this latent space. We demonstrate that SCoTS significantly improves the performance and generalization capabilities of diffusion planners on offline goal-conditioned benchmarks requiring stitching and long-horizon reasoning. Furthermore, augmented trajectories generated by SCoTS significantly improve the performance of widely used offline goal-conditioned RL algorithms across diverse environments.

State-Covering Trajectory Stitching for Diffusion Planners
Kyowoon Lee KAIST leekwoon@kaist.ac..tt
Jaesik Choi KAIST, INEEJI jaesik, choi thoi@kaist... ac. ac.kr
Abstract
Diffusion-based generative models emerging as powerful tools for long-horizon planning reinforcement learning (RL), particularly with offline datasets. How- ever, their performance is fundamentally limited by the quality and diversity of training data. This often restricts their generalization to tasks outside their training distribution or longer planning horizons. To overcome this challenge, we propose State-Coverite Covering Trajectory Stitching (SCoTS), a novel reward-fire free trajectory aug- mentation method that incrementally stitches together short trajectory segments, systematically generating diverse and extended trajectories. SCoTS first learns temporal preseecingg latent representation that captures the underlying temporal structure of the environment, then iteratively stitches trajectory segments guided by directional exploration and novelty effectively cover and expand this latent space. We demonstrate that SCoTS significantly improves the performance and generalization capabilities of diffusion planners offline goal-conditioned benchmarks requiring stitching and long- oong-horizon reasoning. Furthermore, aug- mented trajectories generated by SCoTS significantly improve the performance of widely used offline goal-conditioned RL algorithms across diverse environments. Our code is available at hhttps: //gi thub. com/ http///eeoooo/os//////ooo
Introduction
In many real-world applications, agents must plan over hundreds of steps, often receiving sparse or delayed feedback until they reach a distant goal. Perfect knowledge of the environment allows powerful planners like MPC (Tassa et al. 2012) and MCTS (Silver et al., 2016 2017) to excel. However, most real-world tasks instead require learning environment dynamics from data. Model- based reinforcement learning (MBRL) (Sutton) 2018) constructs such world models, offering sample- efficient learning and improved generalization (Ha & Schmidhuber 2018 Hafner et al. 2019 Kaiser al. 2020). However, autoregressive predictions from learned models accumulate small errors into cascade of inaccuracies. This compounding eтror can cause planners to exploit model inaccuracies and generate trajectories that are suboptimal or even physically infeasible, especially in long-horize
tasks (Talvitie 2014 Asadi et al., 2018, 22018,0ama Janner et al. 2019 Voelcker et al. 2022 Chen et al. 2024a).
To address these limitations, diffusion planners (Janner al. 2022
Ajay et al.
2023 Liang al.
2023; Chen et al. (224ch have recently emerged as a promising alternative for trajectory generation step time, diffusion planners treat each trajectory as a single high-dimensional sample, learning a denoising process that transforms noise drawn from a simple prior into trajectories that match the target distribution (Ho al.) 2020 Song et al. 202T). By operating on entire trajectories simultancously, these methods inherently prevent the compounding of prediction errors that undermine autoregressive dynamics models. Moreover, the generative nature of diffusion models allows for flexible conditioning and guidance mechanisms, enabling the synthesis of plans with properties like reaching specific goals maximizing expected returns DDhariwal & Nichol 2021).
sequential decision-making Instead of rolling
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Despite these advantages, the effectiveness of diffusion planners remains fundamentally lim-
(a) Offline Data
(b) HD
(c) Ours
ited by the quality, diversity, and coverage of the offline training data. First, their effective plan- ning horizon is inherently coupled to the maxi- mum trajectory length observed during training, making challenging to generate coherent plans that significantly exceed this length. Second, their generalization capability is often confined to the specific types of trajectories and transi- tions represented in the training data. For in- stance, if the dataset predominantly features cer- tain movement patterns, the planner may strug- gle to synthesize solutions for novel tasks re- quiring different compositions of behaviors (as illustrated in Figure D. While exhaustively col- lecting data for all conceivable scenarios could mitigate this, such an approach is prohibitively expensive. Trajectory stitching (Ziebart al. 2008) offers a promising alternative by com- posing novel, longer sequences from existing short segments. However, existing methods rely heavily on extrinsic rewards for segment selec- tion, and maintaining the dynamic consistency and feasibility of stitched trajectories remains challenging.
Figure 1: Improved generalization with SCoTS. (a) Examples from the training dataset, illustrating limited coverage. (b) Plans generated by Hierar- chical Diffuser (HD) (Chen et al. (024c), which fail to generalize well these out-of out-of-distribution tasks due to insufficient coverage of the training data. (c) Plans generated by HD trained on SCoTS- augmented data, demonstrating significantly im- proved trajectory stitching capability and general- ization to unseen tasks. Each color corresponds to one of 10 plans generated by the planner.
In this paper, we propose State-Covering Trajectory Stitching (SCoTS), a reward-free trajectory augmentation framework that systematically extends trajectories to cover diverse, unexplored regions of the state space. Specifically, SCoTS employs three-stage approach: First, learn a temporal -ereerripp latent representation by training a model to encode states based on learned optimal temporal distances, facilitating efficient identification of viable trajectory segments. Second, we introduce a novel iterative stitching strategy that balances directed exploration with state-space coverage. In this process, trajectory segments are selected based on their progress along a learned direction in the latent space and their novelty relative to previously explored regions within the rollout. Finally, we refine the resulting stitched trajectories using a diffusion-bart based refinement procedure. Consequently, the resulting trajectories exhibit broader state-space coverage while preserving dynamic feasibility.
To summarize, our contribution in this paper is the introduction of SCoTS, reward-free trajectory augmentation approach designed to generate diverse, bong-horizon trajectories that enhance diffusion planners. Extensive experiments across diverse and challenging benchmark tasks show that SCoTS significantly enhances the stitching capabilities and long-horizon generalization of diffusion planners. Furthermore, augmented trajectories generated by SCoTS notably boost the performance of widely used offline goal-conditioned reinforcement learning (GCRL) algorithms across multiple trajectory stitching benchmarks.
2 Planning with Diffusion Models
Diffusion-based planners (Janner et al. 2022 Liang et al. 2023 Chen et al., 2024c) provide promising framework for long-horizon decision decision-making by modeling entire trajectories as joint distributions. A trajectory is typically represented sequence of states 8, and actions a, planning horizon T:
82
8T
a1
a2
ar
where and ag denote the state and action at time step I, respectively. Diffusion planners utilize diffusion probabilistic models (Sohl-Dickstein et al. 2015 Ho et al. (020)) learn a trajectory distribution po (=())) over noise-free trajectories T". This involves predefined forward noising process

---

denoising steps
(a) Search
(b) Select
(c) Stitch
Figure 2: Overview of the SCoTS stitching process. (a) Temporal Mistance-Preservingg Search:
Given the currently composed trajectory (red), we identify candidate segments (gray) by searching in latent space learned to preserve temporal distances. Candidates selected based on proximity the endpoint of the current trajectory in latent space. (b) Exploratory Segment Selection: Among the retrieved candidate segments, select the segment (blue) that best balances directional progress toward a randomly sampled latent direction and novelty relative to previously visited states in latent space. (c) Diffusion-basee Stitching Refinement: To ensure smooth transitions, diffusion model refines the stitching point between segments, generating dynamically consistent trajectories.
and learned reverse denoising process. The forward process incrementally adds Gaussian noise to the trajectories through M discrete diffusion timesteps with a variance schedule {3, 1M
'(T'(>²-1) == W / - 3,r" 3.1.)
key property the direct sampling of intermediate trajectories:
<(7' | = Va,""
where a; = II' 8.). The schedule ensures that TM approximates a standard Gaussian distribution N (0, I). The reverse process learns invert this noising process and define following generative process with standard Gaussian prior p(T
M
po(TT) p(T II
f
po p 1:M
with learnable Gaussian transition: Po Po N(T (( (HO(') E').
Given an offline dataset D, diffusion models in practice simplify training by parameterizing a noise- prediction network ca, trained predict the noise added during the forward process (Ho al.) 2020
[()) == e |||e [||'-< co(T"",))1]),
(5)
where E (0, MJ } is the diffusion timestep, NO I) is target noise that was used to corrupt clean trajectory - into T' = va5T + /1 açç
Remark. Previous works generally assume the offline dataset D sufficiently covers diverse trajectories with substantial length. Consequently, these studies have primarily focused on improving network architectures, action generation methods, and planning strategies. In contrast, we explicitly aim to generate augmented dataset Dang that extends trajectory coverage, enabling diffusion planners to generalize effectively beyond their training distribution.
3 State-Covering Trajectory Stitching
We introduce State- Cavee-Coveiing Trajectory Stitching (SCoTS), a novel reward-free trajectory augmenta- tion framework designed to synthesize an augmented dataset Doug from an offline dataset D. The core idea of SCoTS is iteratively construct long and diverse trajectories by repeatedly stitching short segments guided by latent directional exploration, resulting in significantly improved generalization and extended planning horizons for diffusion planners. SCoTS consists of three stages: (1) learning
temporal distance-press preservvig embedding for efficient segment retrieval (Section 3.1: (2) iterative
trajectory stitching driven by latent directional exploration and novelty-based selection (Section 3.2
and (3) diffusion- based refinement to ensure dynamically consistent transitions (Section 3.3 The
overall procedure SCoTS, including segment search, exploratory selection, and diffusion-based refinement, is illustrated in Figure The detailed algorithm is summarized in Algorithm

---

Algorithm Overview of the SCoTS Framework
1: Input: Offline dataset D, Temporal distance-preserrinng embedding o, Diffusson stitcher piilchhrr
2: Initialize: Augmented dataset Daug =
3: for n 1,...,,.. Nurg do
4: // Sample initial segment from offline data
5: Tcomp ~ ~
6: // Sample a random latent exploration direction
7:
2 ~~ (O,, I);
+ 2/||=|| 2+
8:
for 1. =1=1,,. 1,....,, 1,..==,1,,,iai.. Nuttch do
9:
// Retrieve nearest segments using temporal embedding
10:
[T (T,))=1 f (ooKNeeigboos((((n(((( commp)), k k)
// Compute directional progress and novelty scores
12:
Compute scores S₁ = P3 + BN,
(Eq. [1I
// Select best candidate segment
14:
Tbest < arg max, Sj
// Diffusion- based stitching refinement
16:
T' Po gilchhe
81 end(Tccmm) commp)
end(Tres))
// Concatenate refined segment to trajectory
18:
Tommp + -T comp: T']
end for
20:
Daug Daag ang U [Tomp]
21:
end for
22:
Train diffusion planner on Daug
3.1 Temporal Distance- Preserving Embedding
Identifying trajectory segments that are suitable for stitching requires accurately measuring their temporal closeness. However, simply using raw state-space distances yield temporally incoherent results due to potential dynamic inconsistencies arising from ignoring state reachability. To address this, employ temporal distance- preservvigg embedding S - Z, which maps states to a latent space designed such that the Euclidean distance ([)(s) - o(g))12 approximates the optimal temporal distance d' g), defined as the minimum number of environment steps required to transition from state 8 to state Formally, we parameterize a goal-conditioned value function (s,, g) following (Park al. 2024a):
V(s,g) = - (0((s (gg))12
(6)
which is trained the offline dataset D using temporal difference objective inspired by implicit Q-learning (Kostrikov et al. 2022):
Ls == (( g)~D 1e(-1(ss
vllō(s') e(g) (lo(s) o(g)]l2)]
(7)
where is target network MMnih 2013). ) 7 is discount factor, and & denotes the expectile loss (Kostrikov et al. 2022; Newey & Powell 1987).
Remark. We note that the learned latent space not a perfect metric representation the MDP (Park et al. 2024a). However, the reliability of SCoTS is grounded in its design, which does not require a globally accurate temporal distance metric. Instead, as we detail next, our framework leverages the latent distance for the more tractable local problem of retrieving promising and reachable candidate segments at each step of the iterative stitching process. This local approach makes the overall framework robust to the inherent imperfections of the embedding.
3.2 Directional and Exploratory Trajectory Stitching
Given the learned temporal distance-preserving embedding Φ we iteratively construct extended trajectories via stitching. We start each new trajectory by randomly sampling an initial segment Timit from the offline dataset D. To encourage diverse state coverage, we randomly sample fixed latent exploration direction unit vector, i.c., (). z/ >||| for each trajectory rollout.

---

At each stitching iteration, let Teomp denote the currently composed trajectory. We define end(T) as function returning the final state of trajectory т. We then identify set of candidate segments (T 1 whose initial states are nearest neighbors to end(Tcomp) in the latent space:
{ (TopKNeiihoorsseeeen(T... (l k),,
(DD),
(8)
where (DD is a concise representation for the set of latent embeddings of the initial states of all trajectories within the dataset D, and the distance metric is () < (811,))l22
To select the best candidate for stitching, we evaluate each candidate segment T, = ($1.) H.)) based on a composite score balancing directional progress and novelty. The progress score quantifies the alignment in the latent space between the segment direction and the exploration direction z:
Pj = (p(end(T,)) - ((51.)).
The novelty score promotes exploration and coverage novel latent states by estimating the entropy of the endpoint of each candidate segment T, relative previously visited latent states. Here, Vooleet denotes the collection of latent representations of every state along previously stitched segments. Leveraging non-parametric particle-based estimator (Liu & Abbeel 202T) on our temporal distance- preserving embeddings, we compute the novelty score as:
Nj- Kdensity
Σ
(e(nn(TT)) Φ
(10)
G.Ek.NN (okeNN((((nd())))) (T))n Voolioat Kaenstoy)
A higher N₂ indicates greater novelty, signaling that the candidate segment expands coverage by moving towards less-explored regions of the latent space. We combine these two metrics to form the overall selection criterion:
S, = Pj 3N,,
(11)
where 3 balances progress and novelty. We then stitch the candidate Tbest with the highest score to Toomp-
3.3 Diffusion-based Stitching Refinement
Although the exploratory selection step identifies segments with desirable progress and novelty, the stitching points, i.e., the connecting states between consecutive trajectory segments, may still exhibit minor dynamic inconsistencies or sub-optimal transitions. diffusion based refinement step. Specifically, we train a diffusion model, termed the stitcher po stithher which generates intermediate states conditioned on the boundary states of adjacent segments. Given selected segment Tbest, the stitcher produces a refined trajectory T' by sampling from:
mitigate these issues, we introduce a
T' ~ pe gitcher(. 81 = end(TTomm)) 8 = end (Thea)))))
(12)
where end(T comp) denotes the end state of the current composite trajectory Teomp- and end(Theu) denotes the end state of the newly selected segment Tbest This diffusion-based refinement effectively smooths out transitions, ensuring dynamic coherence and feasibility of the stitched trajectories.
By iteratively repeating segment search, exploratory selection, and this refinement process, we construct diverse set of augmented trajectories. To generate corresponding action sequences for these trajectories, we train an inverse dynamics model a = (st s(+1) ) on the offline dataset D, which infers the actions that transition between consecutive states. The resulting aate-nctioon trajectories are aggregated into the augmented dataset Dang. This systematic and iterative augmentation approach generates augmented dataset that broadly covers the state space. Crucially, diffusion planners trained on this augmented data exhibit significantly enhanced trajectory stitching capabilities and improved long-horizon generalization, particularly for tasks requiring extensive trajectory stitching and long-hood morgoooizon reasoning (Section
Experiments
In this section, we empirically validate the effectiveness of our proposed SCoTS framework. Specif- ically, we aim to investigate (1) whether SCoTS can generate diverse trajectories that extend sig- nificantly beyond the planning horizons present in the original offline dataset, (2) whether training
5

---

diffusion planners on these augmented trajectories enhances their capability to produce feasible long- morgoorizon plans in unseen scenarios, and (3) whether the augmented dataset generated by SCoTS provides significant performance improvements for existing offline goal-conditioned reinforcement learning (GCRL) algorithms. Additional results can found in Appendix C
4.1 Datasets and Environments
We evaluate SCoTS on OGBench benchmark (Park et al. 2025), spanning diverse difficulties, environment sizes, agent state dimensions, and training data qualities. Specifically, the benchmark includes three locomotion environments: PointMaze (controlling a 2D point mass) and AntMaze (controlling 8-DoF quadrupedal Ant). We consider two distinct dataset types, each designed to evaluate specific challenges. The Stitch dataset comprises short, goal-reaching trajectories limited four cell units, thus requiring the agent stitch multiple segments (up to 8) for successful inference. In contrast, the Explore dataset assesses learning navigation behaviors from extensive yet quw-quality exploratory trajectories, collected by frequently resampling random directions and injecting significant action noise. For each environment, we report the success rate averaged over all evaluation episodes, where an episode is considered successful if the agent reaches sufficiently close to the goal state within a predefined distance threshold. See Appendix A for dataset details.
4.2 Diversity and State Coverage Analysis
stitching iterations
To investigate whether SCoTS effectively pro- motes diverse state-space coverage through tra- jectory stitching, we evaluate its performance in the PointMaze-Giant Stttstttcch environment. As illustrated in Figure 3
B 0
we visualize the incre- mental stitching process for different values of the novelty weighting parameter / {0, 2, 20). We observe that when 3 O, trajectory stitch- ing predominantly follows latent directional guidance, resulting in trajectories with limited coverage but clear directional distinctions. With a moderate setting 3 = 2, trajectories exhibit balanced trade-off, achieving substantial state- space coverage with notable diversity. Con- versely, at higher novelty weight / 20, trajectories broadly cover the state space but lose their distinctiveness, leading to overlapping paths across different latent exploration direc- tions. Based on these results, we use 5 across all environments in our experiments.
β =
A 20
Figure 3: Effeet of novelty score on Trajectory Stitching. Trajectory stitching examples in the SaattMaze-Ciant-Stee
environment. The original dataset (Stitch) consists of short seg- ments limited to at most four maze cells. Different colors represent trajectories generated from dis- tinct latent exploration directions z.
4.3 Diffusion Planning with SCoTS Augmented Data
We next demonstrate how SCoTS- generated trajectories enhance the ability of diffusion planners to generate feasible, long-horizon plans beyond their training distribution. We compare our approach with offline goal-conditioned reinforcement learning (GCRL) methods including goal-conditioned
implicit Q-learning (GCIQL) (Kostrikov al. 2022), Quasimetric RL (QRL) (Wang et al. 2023). Contrastive RL (CRL) (Eysenbach et al.] [022), ), and Hierarchical implicit Q-learning (HIQL. (Park
ct al. 2023). We also include diffusion- long-horizon generalization, such Generative Skill Chaining (GSC) (Mishra et al. 2023 and
generative planning basclines explicitly designed for
Compositional Diffuser (CD) (Luo et al. 2025).
For our experiments, we adopt hierarchical diffusion planner (HD) CCen et al. 2024c) that generates plans through a two-level planning process. Specifically, the high-level diffusion model first generates sparse, temporally coarse waypoints, after which a low-level diffusion model fills in the intermediate states between these waypoints, producing a temporally dense trajectory. Initially constrained by limited and short-horizon training data, we augment the original dataset with SCoTSS-eenerted trajectories. After dataset augmentation, we train diffusion planner and employ a value-based low-
level controller for action execution, following recent approaches (Yoon et al. 2025a Lu al.

---

Table I: Quantitative results locomotion tasks in OGBench. Results averaged over random seeds, each with 50 episodes per task. Standard deviations are reported after the + sign.
Env
Type
Size
GCIQL
QRL
CRL
HIQL
GSC
CD
HD
SCoTS
Kedium
21 19
80 112
74
10010
100.0
2413
100
PointMaze
Stitch
Large
31 42
84 #15
13
100+
10040
1742
100=0
Giant
50
00
2943
6843
0=e
100=0
Kedium
29 +6
59
53 +6
94
97+2
96+2
71+1
97+1
Stitch
Large Giant
18
11 1:2
67
6612
8612
3612
9311
AntMaze
0 0.10
0
.12
20.11
6513
0_0
87.12
Medium
13 +2
1 =1
3 =2
37 =10
90=2
81
42+3
99=1
Explore
Large
0 4:0
0 4:0
0 =00
4 4.5
2143
27
1342
98
Average
12.6
36.5
8.4
36.4
65.3
77.9
25.4
96.8
denoising steps
Ant AntMane-C Glant AntMaza- Larzz-Laar Large
Exploon
Siftch
Figure 4: SCoTS enables long- horizon planning. We visualize trajectories generated by diffusion planner trained on SCoTS-augmented data, evaluated on two challenging AntMaze datasets: Explore (top) and Stitch (bottom). The original Stitch dataset contains trajectories limited to four maze cells per segment, necessitating extensive stitching, whereas the Explore dataset comprises low- quality trajectories with large action noise. Despite these constraints, SCoTS augmentation allows the planner to synthesize trajectories that substantially surpass the horizon and quality of the original data, connecting specified start and goal
2025). The plans generated by the diffusion planner serve as sequences of subgoals for the low-level controller. At each step, the low-level controller executes actions toward subgoal selected from the generated plan; after a fixed horizon or once the subgoal is reached, it dynamically updates the subgoal by selecting the next state at specified horizon further along in the plan generated by the diffusion planner. For each dataset, we upsample the original data 5M samples. Additional implementation details, including hyperparameters and specifics of the low-level controller, are provided in Appendix
As shown in Table ! integrating SCoTS consistently enhances the performance of the hierarchical diffusion planner across all tasks, achieving near-optimal success rates. Notably, the advantage of SCoTS becomes especially pronounced as the complexity and scale of the mazes increase, with the gap between SCoTS and other baselines maximized in the largest (Giant) environments. Furthermore, in the challenging Explore dataset of the AntMaze environment consisting of noisy and short-range exploratory trajectories, augmentation via SCoTS significantly improves the planner ability to generate coherent, long-range goal-directed plans, clearly highlighting the effectiveness of SCoTS.
4.4 Offline GCRL with SCoTTS-uugmented Data
Although SCoTS primarily designed for diffusion planners, we additionally evaluate whether trajectories augmented by SCoTS can enhance the performance of existing offline goal-conditioned RL (GCRL) algorithms. Specifically, we retrain widely used offline GCRL algorithms, including GCIQL (Kostrikov et al. (022). CRL (Fysenbach et al. 2022). and HIQL (Park et al. (023), on the SCoT'S Saagmeettdd dataset. All hyperparameters remain identical their original implementa- tions. Additionally, compare our approach with SynthER (Lu et al. 2023). which employs an

---

Table 2: Performance enhancement of offline GCRL algorithms with SCoTS-augmented dataset. Results are averaged over 5 seeds, each with 50 episodes per task. Standard deviations are indicated by = sign.
GCIQL
HIQ
Env
Type
Size
Original
SynthER
SCoTS
Original
SynthER
SCaTS
Original
SynthER
SCoTS
Medium Large Giant
21 a2
30 2a
79 g1
D 11
0 0.
46 1:2
74 16
77 24
82 1:4
PointNaza
an
35 14
26 22
ao
30 1.
16
16 13
67 11
0 10
1.
0.
0 0.
0.
18 52
0 0.29
0.10
27 122
un
29 26
31 aa
35 22
53 20
48 aa
65 22
94 21
91 29
94
Stitch
1.22
3.14
7.11
11
12
19.11
67
65
91 12
AntMaze
Giant
0 to
+0
0.00
0 00
0.00
2:43
2:
0.00
53 #5
Explore
Mndiuum Large
18:33
1:00
18
94 11
0 .10
0.
0.19
0 0.
2:11
19 21
4
12
77 122
Average
12.6
13.9
20.7
8.4
8.1
27.9
36.4
38.3
73.4
unconditional diffusion model for transition-level data augmentation. Results summarized in Table clearly demonstrate that (CoTS--enerain trajectories consistently outperform SynthER and methods trained solely on the original offline datasets, significantly boosting performance across all tested algorithms. This indicates that augmenting data the trajectory-level with SCoTS, which explicitly considers long-term dynamics and diversity, provides more effective supervision for learning robust trajectory stitching and long-horizon planning capabilities.
4.5 Ablation Studies
AntMazo-Giant Giant-Stitch
100
1000
Without Refireemnnt
SCaT8
Number of Samples
With Refivement
Success Rate (%)
90
CD
800
83
600
80
75
400
65
200
60
55
10
15
20
25
10-2
Low-level Controller Horizon
Dynamic MSE
Ablation study on low-level controller horizon. Success rates the environment compar- ing SCoTS against Compositional Diffuser (CD) (Luo al. 2025), across various low-level con- troller horizon lengths.
Figure 5:
Figure 6: Dynamic MSE comparison at stitch- ing points. Histograms showing the distribu- tions of Dynamic MSE at trajectory stitching points in the NntMaze-Giant--a ment, comparing results with and without the diffusion-basse stitching refinement step.
AntMaze- AntMaze-Giant-Ste
environ-
Ablation study low-level controller horizon. We investigate how the performance of our approach (SCoTS) is influenced by varying the horizon length of the low-level controller in the AntMaze-Gi ant-antttttt environment. As shown in Figure 5, SCoTS achieves consistently strong performance across different horizon lengths H E (5, 10, 15, 20, 25}. outperforming the Composi- tional Diffuser (CD) (Luo et al. 2025). These results demonstrate that the diffusion planner trained with SCoTS generates highly feasible subgoals, maintaining robustness and effectiveness regardless of the chosen low-level execution horizon.
Effectiveness of diffusion-hased stitching refinement. To further illustrate the effectiveness of the diffusion-based stitching refinement step in our SCoTS framework, we quantitatively evaluate its impact on dynamic consistency at stitching points. Specifically, we compute the Dynamic Mean Squared Error (Dynamic MSE) (Lu et al. 2023), defined as:
Dynamic MSE (s, s'112.

---

which measures how closely the generated transitions adhere to the true environment dynamics f- Figure [ compares the distribution of Dynamic MSE at stitching points before and after applying refinement on logarithmic scale. Results clearly show that diffusion- based refinement substantially reduces dynamic inconsistencies, highlighting its critical role generating dynamically feasible and coherent trajectories.
Ablation study on replanning. We em- ploy replanning during a rollout, enabling the agent to recover from failures, such as when the diffusion planner generates un- reachable subgoals for the low-level con- troller. In practice, we set a replanning in- terval (e.g., every 200 steps); further im- plementation details are provided in Ap- pendix B In Table E] present abla- tion study comparing performance with and without replanning on the PointMaze and AntMaze Sti datasets from OGBench. SCoTS consistently outperforms Composi- tional Diffuser (CD) (Luo et al.) 2025), the best-performing baseline, even without re- planning. Additionally, the performance with and without replanning is similar, highlighting the reliability and efficacy of the (CoTSS-augment diffusion planner.
Table 3: Impact of replanning. Success rates on OGBench PointMaze and AntMaze Stitch datasets, comparing SCoTS and CD (Luo et al. 2025). of indi- cates with replanning; X indicates without replanning.
CD
SCoTS
Env
Size
x
v
X
/
Medium
100 =o
100 =0
100 =o
100 =0
PointMaze
Large
100 1o
100 10
100 10
100 10
Giant
53 26
68 23
89 22
100 20
Medium
92
96 122
97 11
97 11
AntNaze
Large
76 +2
86 +2
92 +2
93 +1
Giant
27 #:4
#2
#2
Average
74.7
85.9
93.7
96.2
5 Related Work
Planning with Diffusion Models. Diffusion probabilistic models (Sohl-Dickstein et al., 2015 Ho al.] 2020) have emerged as powerful tools for reinforcement learning, especially offline settings. These models iteratively denoise sampled data from noise, effectively learning gradients of the data distribution (Song & Ermon 2019) and demonstrating strong capabilities in modeling complex trajectories. Early work such as Diffuser (Janner et al., (202) employed unconditional diffusion models guided by learned value estimators (Dhariwal & Nichol (021). Subsequent methods like Decision Diffuser (Ajay et al. 2023) and AdaptDiffuser (Liang et al. 2023) introduced classifier- free guidance and progressive fine-tuning. Recent advancements further leveraged hierarchical structures (Chen et al. 2024c] Li et al., 2023), multi-task conditioning (Ni et al.) [0233 He et al.) 2023; Dong et al. 2024), and multi-agent setups (Zhu et al., 2023). Additionally, diffusion planners have explored integration with tree search methods ( Yoon et al. 2025a b), refined trajectory sampling techniques (Lee et al. 2023b Feng et al.
2024 Lee & Choil 2025). and investigated critical design choices improve robustness (Lu al. (025). Despite these advances, diffusion planners still fundamentally depend on the quality and diversity of the offline training datasets, limiting their ability generate coherent and feasible long-horizon plans beyond their training distribution. Recent approaches such as Generative Skill Chaining (GSC) (Mishra et al., 2023) and Compositional Diffuser (Luo al. 2025) address this by composing short segments at test time into long-horizon trajectories. Our work presents an orthogonal solution by directly augmenting the offline dataset itself, significantly enhancing the capability of diffusion planners to generalize to diverse and substantially longer trajectories.
Data Augmentation for RI.. Data augmentation is a recognized strategy for improving sample efficiency and generalization in reinforcement learning (RL). In pixel-based RL, techniques like random image transformations (e.g., cropping, translation) have proven effective in works such
as CURL (Laskin et al.) 20206), RAD (Laskin al., 2020a), and DrQ (Yarats et al. 202T). For state- based observations, methods like S4RL (Sinha et al.) 2022) and AWM (Ball et al. 2021) often introduce perturbations to states or learned dynamics models to enhance robustness. Recent advances in generative models have enabled trajectory-Ieve augmentation methods, either at the transition 2023 Wang et al. 2024) or the full trajectory level (He et al. 2023 Jackson et al.. 2024: Lee et al. 2024 For instance, MTDiff-S S (He et al.) 2023) generates synthetic trajectories for
level (Lu et al.
multi-task scenarios, while Policy-Guided Diffusion (PGD) (Jackson et al. 2024) and GTA (Lee et al.,
2024) employ generative models to produce high-rewart trajectories guided by policies or returns.

---

DiffStitch (Li et al. 2024) further systematically connects trajectories based on extrinsic rewards, yet these methods typically require explicit reward signals and are limited to generating short-horizon trajectories. In contrast, our proposed SCoTS method operates in a rewarl-free manner, systematically synthesizing long-horizon, diverse, and dynamically consistent trajectories to significantly enhance offline datasets, thereby facilitating the generation of feasible plans in downstream tasks requiring extended horizon reasoning.
Temporal Distance in RL. Temporal distance has been widely adopted as structural inductive bias in various reinforcement learning (RL) paradigms, including imitation learning (Sermanet
et al. 2018), unsupervised skill discovery (Hartikainen et al.) [2019] Park et al. 2024b a), goal- conditioned RL (Durugkar et al.] 2021; Eysenbach et al. 2002 Wang et al.) 2023 Bae et al.) 2024), and curriculum learning (Zhang et al.] 2020 Kim et al. (023). Recent methods such as METRA PPrk et al. (2024b), QRL (Wang et al. 2023), HILP (Park et al. 2024a), and TLDR (Bac) ct al. 2024) particularly focus on learning temporal
representations to facilitate diverse skill discovery or efficient goal-reaching behaviors. Distinct from prior methods, our SCoTS framework explicitly leverages temporal distance-preserving representations identify temporally viable trajectory segments for stitching. This allows systematic synthesis of extended, diverse, and dynamically consistent trajectories, significantly augmenting offline datasets and improving long- mori-horzzon generalization for diffusion- based planners.
6 Conclusion
In this work, we introduced State- Covering Trajectory Stitching (SCoTS), a novel reward-free trajec- tory augmentation approach designed to enhance the performance and generalization capabilities of diffusion planners. By leveraging temporal distance-preserving preserving embeddings, SCoTS iteratively stitches together short trajectory segments, systematically extending the diversity and horizon of offline data. Empirical results across challenging benchmarks demonstrated that SCoTS--eenrann tra- jectories significantly improve the ability of diffusion planners to perform long-horizon planning and generalize to novel tasks. Furthermore, we showed that our augmented dataset notably enhances the performance of widely used offline goal-conditioned reinforcement learning algorithms, highlighting the broad utility of our approach.
Limitations. While SCoTS achieves strong empirical performance, it exhibits certain limita- tions. First, generating augmented trajectories through iterative stitching and cirtuusionbbasee re- finement introduces significant computational overhead, especially due to the additional training of the ieffusiionbbsed stitcher model and the trajectory augmentation process. Second, our temporal distance- prreeervnnn embeddings do not capture the asymmetric temporal distances between states, potentially limiting their effectiivnees in highly asymmetric or disconnected Markoy Decision Pro- cesses (MDPs), such as object manipulation tasks involving irreversible actions or environments containing isolated regions with sparse connectivity.
Impact Statement.
This paper advances the field of diffusion based planning by introducing a novel trajectory augmentation method, enhancing long-horizon reasoning capabilities in reinforcement learning. While we do not identify direct negative societal impacts stemming from this research, practitioners are encouraged carefully assess real- world implications, particularly regarding safety and reliability, prior to deploying this method in practical scenarios.
Acknowledgements
This work was supported by Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) (N0.2019-0-00075. Artificial Intelligence Graduate School Program (KAIST); No. 2022-0-00984, Development of Artificial Intelligence Technology for Personalized Plug-and-Plas Explanation and Verification Explanation; No. RS-2024-00457882, AI Research Hub Project; No. 20222-2022-11 84, Development and Study of AI Technologies to Inexpensively Conform to Evolving Policy on Ethics), and by the InnoCORE program of the Ministry of Science and ICT (N10250156).
10

---

References
Ajay, A., Du, Y., Gupta, A., Tenenbaum, J. B., Jaakkola, T. S., and Agrawal, P. Is conditional generative modeling all you need for decision making? In International Conference Learning Representations (ICLR), 2023.
Asadi, K., Misra, D., and Littman, M. Lipschitz continuity in model-based reinforcement learning.
In International Conference on Machine Learning, pp. 264-273. PMLR, 2018.
Bae, J., Park, K., and Lee, Y. Tidr: Unsupervised goal-conditioned rl via temporal distance-aware representations. arXiv preprint arXiv:2407.08.. 0..444.0444. 2024.
Ball, P. J., Lu, C., Parker-Hoider, J., and Roberts, S. Augmented world models facilitate zero- shot dynamics generalization from single offline environment. In International Conference on Machine Learning (ICML), 2021.
Chen, B., Martí Monsó, D., Du, Y., Simchowitz, M., Tedrake, R., and Sitzmann, V. Diffusion foreing: Next-token prediction meets full- sequence diffusion. In Advances Neural Information Processing Systems (NeurIPS), 2024a.
Chen, C., Baek, J., Deng, F., Kawaguchi, K., Gulcehre, C., and Ahn, S. Plandq: hierarchical plan orchestration via d-conductor and q-performer. arXiv preprint urXiv:2406.06793 2024b.
Chen, C., Deng, F., Kawaguchi, K., Gulcehre, C., and Ahn, S. Simple hierarchical planning with diffusion. arXiv preprint arXiv:2401.0266 2024c.
Dhariwal, P. and Nichol, A. Diffusion models beat gans image synthesis. In Advances in Neural Information Processing Systems (NeurIPS). 2021.
Dong, Z., Yuan, Y., Hao, J., Ni, F., Mu, Y., Zheng, Y., Hu, Y., Lv, T., Fan, C., and Hu, Z. Aligndiff: Aligning diverse human preferences via behavior-customisaale diffusion model. arXiv preprint arXiv:2310.0206 2023.
Dong, Z., Hao, J. Yuan, Y. Ni, F., Wang, Y., Li, P., and Zheng, Y. Diffuserlite: Towards real-time diffuusin planning. Advances in Neural Information Processing Systems (NeurIPS), 2024.
Douze, M., Guzhva, A.. Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L.. and Jégou, H. The faiss library. arXiv preprint arXiv:2401.08281, 2024.
Durugkar, I., Tec, M., Niekum, S., and Stone, P. Adversarial intrinsic motivation for reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Eysenbach, B., Zhang, T., Levine, S., and Salakhutdinov, R. R. Contrastive learning as goal- conditioned reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Feng, L., Gu, P., An, B., and Pan, G. Resisting stochastic risks in diffusion planners with the trajectory aggregation tree. arXiv preprint 1rXhv:2405.17879, 2024.
Ha, D. and Schmidhuber. J. World models. arXiv preprint arXivv1803.10111 2018.
Hafner, D., Lillicrap, T., Fischer, I,, Villegas, R., Ha, D., Lee, H., and Davidson, J. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning (ICML), 2019.
Hartikainen, K., Geng, X., Haarnoja, T., and Levine, S. Dynamical distance learning for semi- supervised and unsupervised skill discovery. arXiv preprint arXiv:1907.080: 08225, 2019.
He, H., Bai, C., Xu, K., Yang. Z., Zhang, W., Wang, D., Zhao, B., and Li, X. Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Higgins, I., Matthey, L.,, Pal, A., Burgess, C., Glorot, X., Borvinick, M., Mohamed, S., and Lerchner, A. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference Learning Representations (ICLR), 2017.
11

---

Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS). 2020.
Jackson, M. T., Matthews, M. T., Lu, C., Ellis, B., Whiteson, S., and Foerster, J. Policy-guided diffusion, arXiv preprint 2444::240, 2024.
Janner, M., Fu, J., Zhang, M., and Levine, S. When to trust your model: Model-based policy
optimization. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Janner, M., Du, Y. Tenenbaum, J. B., and Levine, S. Planning with diffusion for flexible behavior synthesis. In International Conference Machine Learning (ICML), 2022.
Kaiser, L.,, Babaeizadch, M., Mitos, P., Osiński, B., Campbell, R. H., Czechowski, K., Erhan, D.,
Finn, C., Kozakowski, P., Levine, S.,, al. Model-basee reinforcement learning for atari. In International Conference on Learning Representations (ICLR), 2020.
Kim, S., Lee, K., and Choi, J. Variational curriculum reinforcement learning for unsupervised discovery of skills. International Conference on Machine Learning (ICML), 2023.
Kostrikov, I., Nair, A., and Levine, S. Offlin reinforcement learning with implicit q-learning. In International Conference on Learning Representations (ICLR), 2022.
Kumar, A., Zhou, A., Tucker, G., and Levine, S. Conservative q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., and Srinivas, A. Reinforcement learning with augmented data. In Advances in Neural Information Processing Systems (NeurIPS), 2020a.
Laskin, M., Srinivas, A., and Abbeel, Curl: Contrastive unsupervised representations for reinforce- ment learning. In International Conference on Machine Learning (ICML), 2020b.
Lee, J., Yun, S., Yun, T., and Park, J. Gta: Generative trajectory augmentation with guidance for offline reinforcement learning. arXiv preprint 2rXiv:2405.16900,, 2024.
Lec, K. and Choi, J. Local manifold approximation and projection for manifold-ane aware diffusion planning, arXiv preprint arXiv:2506. 00867, 2025.
Lee, K., Kim, S., and Choi, Adaptive and explainable deployment of navigation skills via hierarchical deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 1673-1679, 2023a.
Lee, K., Kim, S., and Choi, J. Refining diffusion planner for reliable behavior synthesis by automatic detection of infeasible plans. Advances in Neural Information Processing Systems (NeurIPS), 2023b.
Li G., Shan, Y.. Zhu, Z., Long, T., and Zhang. W. Diffistith:: Boosting offline reinforcement learning with diffusion- based trajectory stitching. In International Conference on Machine Learning (ICML), 2024.
Li, W., Wang, X., Jin B., and Zha, H. Hicrarchical diffusion for offline decision making. In International Conference on Machine Learning (ICML), 2023.
Liang, Z., Mu, Y., Ding, M., Ni, F., Tomizuka, M., and Luo, P. Adaptdiffuser: Diffusion models as adaptive self-evolving planners. In International Conference on Machine Learning (ICML), 2023.
Liu, H. and Abbeel, P. Behavior from the void: Unsupervised active pre-training. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Lu. C., Ball, P., Teh, Y. W., and Parker-Holder, J. Synthetic experience replay. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Lu, H., Han, D., Shen, Y., and Li, D. What makes a good diffusion planner for decision making? In International Conference Learning Representations (ICLR), 2025.
12

---

Luo, Y., Mishra, U. A.. Du, Y., and Xu, D. Generative trajectory stitching through diffusion composition. arXiv preprint WXXi::200300513,, 2025.
Mishra, U. A., Xue, S., Chen, Y., and Xu, D. Generative skill chaining: Long-horizon skill planning with diffusion models. In Conference on Robot Learning. pp. 2905-29029999 2925.. PMLR, 2023.
Mnih, V. Playing atari with deep reinforcement learning. arXiv preprinl arXiv:1312.5500 2013.
Newey, W. K. and Powell, J. L. Asymmetric least squares estimation and testing. Econometrica: Journal of the Econometric Society, pp. 819-847, 1987.
Ni, F., Hao, J., Mu, Y., Yuan, Y., Zheng, Y., Wang, B., and Liang, Z. Metadiffuser: Diffusion model as conditional planner for offline meta-rl. In International Conference on Machine Learning (ICML), 2023.
Park, S., Ghosh, D., Eysenbach, B., and Levine, S. Hiql: Offline goal-conditioned with latent states as actions. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Park, S., Kreiman, T., and Levine, S. Foundation policies with hilbert representations. International Conference on Machine Learning (ICML), 2024a.
Park, S., Rybkin, O., and Levine, S. Metra: Scalable unsupervised rl with metric-aware abstraction. In International Conference on Learning Representations (ICLR), 2024b.
Park, S., Frans, K., Eysenbach, B., and Levine, S. Ogbench: Benchmarking offline goal-conditioned rl. In International Conference on Learning Representations (ICLR), 2025.
Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings the IEEE/CVF international conference on computer vision, pp.
the
2023.
Sermanet, P., Lynch, C., Chebotar, Y., Hsu. J., Jang, E., Schaal, S., Levine, S., and Brain, G. Time- contrastive networks: seef-super learning from video. 2018 IEEE international conference robotics and automation (ICRA), pp. 1134-1141, IEEE, 2018.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Dricssche, G., Schrittwieser, J., Antonoglou, I., Panncershelvam, V., Lanctot, M., et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484-489 2016.
Silver, D., Schritwwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. Mastering the game go without human knowledge. nature, 550 (7676):354-359, 2017.
Sinha, S., Mandlekar, A., and Garg, A. $4rl: Surprisingly simple self-supervision for offline reinforcement learning in robotics. In Conference on Robot Learning, pp. 907-917. PMLR, 2022.
Sohl-Dickstein, J.. Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), 2015.
Song. J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint 202:02000. 2020.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations (ICLR), 2021.
Sutton, R. S. Reinforcement learning: An introduction. A Bradford Book, 2018.
Talvitie, Model regularization for stable sample rollouts. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2014.
13

---

Tassa, Y., Erez, T., and Todorov, E. Synthesis and stabilization of complex behaviors through online trajectory optimization. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 19906-4913, 2012.
Voelcker, C., Liao, V. Garg. A., and Farahmand, A.-m. Value gradient weighted model-based reinforcement learning. In International Conference on Learning Representations (ICLR), 2022.
Wang, R., Frans, K., Abbeel, P., Levine, S., and Efros, A. A. Prioritized generative replay. arXiv preprint arXiv:2410.18085 2024.
Wang. T., Torralba, A., Isola, P., and Zhang, A. Optimal goal-reaching reinforcement learning via quasimetric learning. In International Conference on Machine Learning (ICML), 2023.
Yarats, D., Kostrikov, I.. and Fergus, R. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. International Conference on Learning Representations (ICLR), 2021.
Yoon, J., Cho, H., Back, D., Bengio. Y., and Ahn. S. Monte carlo tree diffusion for system planning. In International Conference on Machine Learning (ICML), 2025a.
Yoon, J., Cho, H., Bengio, Y., and Ahn, S. Fast monte carlo tree diffusion: 100x speedup via parallel sparse planning. arXiv preprini arXiv:2506.0944 2025b.
Zhang, Y., Abbeel, P. and Pinto, Automatic curriculum learning through value disagreement. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Zhu, Z., Liu, M., Mao, I.., Kang, B., Xu, M., Yu, Y., Ermon, S., and Zhang, W. Madiff: Offline multi-agent learning with diffusion models. arXiv preprint arXiv:2305. 17330, 2023.
Ziebart, D., Maas, A. L., Bagnell, J. A., Dey, A. K., et al. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pp. 1438-1438. Chicago, IL, USA, 2008
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions and scope of our approach, supported by Sections and
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations regarding computational cost and assumptions in the temporal embeddings are discussed explicitly the Conclusion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
The paper does not include theoretical proof' or results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Experimental details, including dataset and implementation specifics, are provided in Sections 4 and Appendix B Our source code to replicate the results is also included in the supplemental materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide source code in the supplemental materials reproduce augmented data and replicate all experimental results reported in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
cublic//uudees//ddeetbbi.ss cesssiinoooyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wwtps/cccppplclgguiieeeeaec
Stteee//CooSSSemsssS.Si.a www.ssssiicggcooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Detailed implementation details, including hyperparameters, are fully provided in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Results are presented with standard deviations computed over random seeds as detailed in Tables and
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources, including types of GPUs and execution times, are detailed in Appendix B
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research complies fully with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts discussed explicitly in the Conclusion section of the paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no risk of misuse as it focuses purely on offline reinforcement learning benchmarks and controlled environments.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We clearly cite and reference the datasets used, such as OGBench.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce new datasets or other assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve human subjects or IRB considerations.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This research does not use large language models as part of its methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /tappp:///eurrs..... httpa//rps.ccccmmeeeerreeee//e/e cc//oonferences/2225///L. for what should or should be described.
21

---

A Details of Datasets
We evaluate our method on the OGBench benchmark (Park et al. 2025 Since our primary goal to assess trajectory stitching capability and long-horizon reasoning, we specifically utilize the Stitch and Explore datasets. As shown in Figure 7 the Stitch dataset is explicitly designed to challenge trajectory stitching ability, comprising short, goal-reaching trajectories limited to a maxi-
Table 4: Dataset specifications.
Episode Length
Euy
Type
Size
Transitions
Epploodes
Kedius
1M
5,000
200
PointMaze
Btitch
Large
IM
5,000
200
Glant
1M
5,000
200
Medius
200
Large
Explore
Modium
Large
SM
SM
10,000
10,000
300
500
mum length of four cell units. Consequently, agents must effectively stitch together multiple short segments (up to eight) to successfully complete long-horizon tasks. contrast, the Explore dataset designed to test navigation skills learned from extensive yet low-quality trajectories. These trajec- tories are generated by commanding a low-level policy with random movement directions re-sampled every steps, along with significant action noise. Each demonstration trajectory typically spans only two to three blocks, resulting in noisy and clustered paths that pose additional challenges for evaluating the ability learn effective policies from highly suboptimal data.
(a) MointMMaze-Medium
(b) PointMaze-Large--e0
(c) PointMaze-Giante Ciant-Stitchh
(d) AntMaze Medium Sertilu-Stitchh
(e) AntMaze Large- -titcch
(f) AntMaze-Giant-Sue
O0
O
(g) MilMazeeMediin un-umm-au -omxxllore
(h) AntMaze Laree-Lrrgg--aae -Expore
Figure 7: Visualization of trajectories from OGBench datasets. Each sub-figure illustrates example
trajectories from different combinations of environments and datasets used our experiments.
B Implementation Details
Network architecture. We utilize DiTID (Peebles & Xie 2023) as the neural network backbone for both the diffusion planner and the stitcher, due to its large receptive field and effectiveness in modeling trajectory-level dependencies. Following prior studies (Dong et al., 2023 Lu et al. (0255) employ DITID architecture with hidden dimension of 256, head dimension of 32, total of 8 DiT blocks consistently across all environments.
22

---

Table Hyperparameters for SCoTS.
Component
Hyperparameter
Value
Tuning Choices
SCoTS: Temporal Distance- Embedding (()
Learning Rate
3 x 10
Latent Dimension
32
Batch Size
1024
Training Steps
1,000,000
Network Backbone
MLP
MLP Dimensions
(512, 512, 512)
Expectile (5 for 2
0.95
SCoTS: Inverse Dynamics Model (for actions in Daug)
Network Backbone
MLP
MLP Dimensions Training Steps
(256, 256, 256)
200,000
SCoTS: Stitching Process Parameters
Top-k Candidates (Search)
10
Raensit (Novelty Score)
30
Novelty Weight (3)
2.0
Augmented Dataset Size
~5 transitions
Nstch (Stitches per Traj.)
Task-dependent (e.g.. 40)
Nusj (Generated Traj.)
Task-dependent (e.g.. 5,000)
SCoTS: Diffusion- Stitcher (p) gitthher)
Network Backbone
DITID
Learning Rate
x 10
Weight Decay
x 10
Batch Size
64
Training Steps
1,000,000
Solver
DDIM
Sampling Steps (DDIM)
20
Horizon (Hscher))
26
Hierarchical Diffusion Planner (HD)
Network Backbone
DITID
Learning Rate
10
Weight Decay
1 10
Batch Size
64
Training Steps
1,000,000
Solver
DDIM
Sampling Steps (DDIM)
20
Plan Horizon (on original data)
101 (Stitch), 401 (Explore)
Plan Horizon (on Dany)
501 (M/1.,, 1001 (G/Explore)
Temporal Jump
26
Execution Parameters
Low-level Controller Horizon
Tuned
{5. 10, 15, 20, 25}
Replanning Interval
Tuned
{50, 100, 2003
23

---

environment, we Details of the low-level controller. A key challenge in diffusion based planning is balancing global trajectory coherence with effective low-level control in high-dimensional state-action spaces (Chen et al., 2024a by Yoon et al., 2025a). Previous approaches, such as PlanDQ (Chen et al.] 2024b) and MCTD Yoon et al. 2025a), address this issue by integrating high-level diffusion planners with separately trained level controllers. Similarly, we adopt hierarchical strategy, where the diffusion planner generates plans based primarily on compact, lower-dimensional state representations (e.g., positions of the agent itself), delegating the fine-grained, low-level action execution to dedicated low-level controller. In our experiments, we specifically employ GCIQL (Kostrikov et al. 2022) as the learned low-level policy in the PointMaze environments and CRL (Eysenbach et al. 2022) in the AntMaze environments. A detailed visualization of generated subgoals and their corresponding execution rollouts can be seen in Figure ? Furthermore, an ablation study examining the impact of the horizon length of the low-level controller is presented in Figure S
Implementation details for SCoTS and diffusion planning. In the temporal presservig search stage of SCoTS, we retrieve the top k 10 candidate segments based on their proximity in the learned latent embedding space during each stitching step. For computing the novelty score, we utilize a density estimator parameter Kdensity = 30 and set the novelty weighting factor / = 2.0 consistently across all tested environments. The horizon length for the diffusion- stitcher is uniformly set to Sticher 26.
To generate the augmented dataset Daug, we perform the stitching procedure Nstitch iterations per trajectory, creating a total of Nusj trajectories. thus ensuring the augmented dataset comprises approximately million transitions. Specifically, in the LtMaaze-Large-Siie set Nincc 40 and Nun- = 5000.
For configuring the Hierarchical Diffusion (HD) planner (Chen et al.) 2024c). parameters are adapted according to the properties of the training data. When training on the original Stitch and Explore datasets, which contain inherently shorter trajectories (as detailed in Table column "Data Episode Length"), we set the high-level planning horizon to 101 steps for Stitch and 401 steps for Explore, both with temporal jumps of 26 steps between waypoints. However, when utilizing SCoTS-augmentee datasets that feature longer and more diverse trajectories, we extend this planning horizon to 501 steps for Medium and Large environments, and 1001 steps for Giant environments, maintaining the temporal jump of 26 steps. Similarly, for SCoTS -augmeete Explore datasets, we also use a planning horizon of 1001 steps with 26-step jumps.
We apply jumpy denoising with DDIM sampling (Song et al., 2020) using 20 denoising steps across all environments. Additionally, we tune the replanning interval from the set (50, 100, 200} steps and tune the horizon for the low-level controller from {5, 10, 15, 20, 25). A full list of the hyperparameters is reported in Table 5
Practical implementation of temporal cestance-preserving search. Our SCoTS framework relies on a Iearned latent space Z where the L2 distance, [[0(s) و(g)) 2. approximates the optimal temporal distance d' d' .) between states (as detailed in Section 3.1.. A critical step in SCoTS is the efficient identification of suitable candidate trajectory segments from a large offline dataset D. This requires fast nearest neighbor search mechanism within the learned latent space 2. To achieve this, we employ an Inverted File (IVF) index from the Faiss library (Douze et al. 2024), which is specifically designed for large-scale similarity searches.
The practical implementation of this search mechanism involves several stages. First, we prepare the data for indexing. This consists of computing the latent embeddings di d(8mm)) for the initial states Sinit of all trajectories within the offline dataset D. Let denote the dimensionality of these latent embeddings. An IVF index is then constructed upon this collection of d-dimensional vectors. The construction process begins by partitioning the latent vectors into Tijstt clusters using the k-means algorithm. Each cluster is represented by a centroid c₁ E {CI,..... Crua J. Subsequently, each latent vector of (ma) in our collection is assigned to its nearest centroid, and for each centroid, an inverted list is maintained, storing references to the vectors belonging to its cluster.
During the temporal distance- preseerin search phase of SCoTS (detailed in Algorithm I line 10), the latent embedding of the current composed trajectory endpoint, (((mnd(Tcomm)), serves as the query vector q. To find the nearest neighbors for q, the IVF index first identifies a limited set of clusters whose centroids {c, } are closest to the query vector q. The search for neighbors is then confined to
24

---

the latent vectors stored within the inverted lists corresponding to these selected clusters. This targeted approach significantly prunes the search space compared to an exhaustive search. Furthermore, the Faiss library provides support for GPU acceleration, which can further expedite this search process and enable efficient candidate retrieval. Once the k nearest latent embeddings corresponding to initial states of segments are identified, we retrieve the full original trajectory segments from ' to form the candidate set for the stitching process.
Computational resources and runtimes. All experiments were conducted using single NVIDIA A10 GPU. The approximate execution times for each component of our method are as follows:
Temporal distance-preservinng embedding training: 1.5 hours
Inverse dynamics model training: 0.25 hours
Low-level controller training: 2.5 hours
Diffusion-based stitcher training: hours
Trajectory augmentation via SCoTS: 0.5 hours
Diffusion planner training: 18 hours
These times per model training instance data generation run and may vary slightly depending on the specific environment and dataset characteristics.
C Additional Results
Ablation study on temporal Distance-preserving embedding. To isolate the contribution of our learned latent space, we compare the performance of the HD (Chen et al. 2024c) planner trained on SCoTS-augmente
Table 6: Ablation on the temporal distance- preserving embedding. Success rates OGBench PointMaze and AntMaze Stitch datasets. indicates stitching guided by raw state-space distances (w/o temporal embed- ding), while / indicates guidance from learned temporal embedding.
data where the stitching pro- cess was guided by either our temporal distance- preserving embedding raw state space. As shown in Table 6 temporal distance-presert tation is crucial for effective stitching. This is espe- cially true in nigh-dimensional environments like AntMaze, where small Euclidean distance in the raw state space (e.g., between two similar joint con- figurations) does not guarantee reachability. Even in Pointmaze, where state-space distance is more intuitive, the learned compact latent space provides a better-structured representation for stitching tem- porally extended trajectories.
represen-
SCoTS
Env
Size
X
Large
93#0
100
PointMaze
Giant
52+1
100+0
Large Giant
45a1
9311
AntMaze
7
8712
Average
49.3
95.0
Ablation study diffusion-based stitching refinement. To demonstrate the importance of the refinement step, we compare the performance of both a diffusion planner (HD) and GCRL al- gorithm (HIQL) trained on SCoT'S data generated with and without the diffusion- based refinement. The results, summa- rized in Table 7 reveal the critical im- portance of this component. Without re- finement, the connection points between stitched segments can suffer from large dynamic inconsistency errors, as illus- trated in Figure 6 This performance degradation partieularly significant when training GCRL algorithms like HIQL, which are highly
Table 7: Ablation on diffusion-based stitching refine-
ment. Success rates on AntMaze Stitch datasets for HD (Chen et al.) (022c) and HIQL (Park et al.) (022)). X indicates training on SCoTS data generated without re- finement, while / indicates training on data generated with refinement.
HD
HIQL
Env
Size
X
Large Giant
85
93
5212
9112
AntMaze
5311
8712
11:2
5515
Average
69.0
90.0
31.5
73.0
sensitive to the dynamic validity of the training transitions.
25

---

Sensitivity analysis on the novelty weight B. As illustrated in Figure using only the progress score (3 = 0), which quantifies the alignment with a latent exploration direction, effective at generating long, semporally-extended trajectories with clear directional distinctions. However, relying solely on this directional guidance can be suboptimal. The learned temporal distance- prreeeenn embedding & is powerful but imperfect approximation of the true environment topology. Due to embedding errors, the geometry of the latent space can be distorted. Consequently, a straight line between two latent states, d(s) and o(s'), may not correspond to a feasible path in the underlying MDP. Therefore, exploratory detours are often necessary to find a valid temporally extended path. This is precisely where the novelty score becomes critical. It encourages these exploratory detours by rewarding the selection of segments that lead to visstvd states. This allows the agent to navigate around the imperfections and distortions in the learned latent space, discovering feasible and often more effective paths. To demonstrate this complementary effect quantitatively, we conducted an additional ablation study 2 OGBench tasks. The table below compares the performance of the HD planner trained with SCoTS data generated using different novelty weights /3 (5 seeds). The results, summarized in Table 8 suggest that while directional stitching alone (3 0) provides notable performance improvement, introducing and balancing it with the novelty score (3 > 0) yields
substantial further gains. This effect is especially pronounced on the more complex Gi ant task.
Table 8: Sensitivity to novelty weight 3.
Env
Type
Size
HD
3=0
B=2
3=4
8=8
Large
3622
8712
9321
9211
8523
AntMaze
Stitch
Giant
0zo
6343
87#2
8942
74
Sensitivity analysis on sub-trajectory length H. As shown in Table performance the Stitch dataset is relatively robust to the choice of H. However, on the Explore dataset, shorter segments perform best. We hypothesize this is because the Explore dataset consists of low-quality, noisy trajectories. Using shorter segments allows SCoTS to be more selective, finding and connecting the temporally-extended parts of trajectories.
Table 9: Sensitivity to sub-trajectory length H.
Env
Type
Size
H=26
H=52
H=104
Stitch
Giant
87+2
89
87+2
AntMaze
Explore
Large
98+1
93+1
88+2
Sensitivity analysis on the number of retrieved segments K. Table shosws that performance robust as long as K is not too small. A very small K limits the diversity of candidate segments, hindering the effectiienees of our progress and novelty-based selection. While performance is stable for larger K, we expect an excessively large K could eventually degrade performance by increasing the chance of retrieving dynamically inconsistent segments.
Table 10: Sensitivity to number of retrieved segments K.
Env
Type
Size
K=3
K=10
K=20
Stitch
Giant
6513
8712
8942
AntMaze
Explore
Large
7212
98=1
97+1
Evaluation on dense-reward offline RL benchmarks. While our paper focuses on tasks where extrinsic rewards are absent, the SCoTS framework not limited to goal-conditioned tasks. To demonstrate its applicability to general, dense-reward offline RI., we conducted additional experi- ments on four standard MuJoCo locomotion benchmarks from D4RL (Kumar et al. (020), utilizing suboptimal medium and medium-reple datasets. It is important to clarify that our use of reward- free refers to the stitching mechanism itself, which is guided by the intrinsic temporal structure the data rather than extrinsic task rewards. This allows the core data augmentation to be task-agnostic.
For fair comparison, we used the same Diffuser architecture and planning horizon (32) as described in the original paper (Janner et al.) 2022). We upsample the original data to 2M samples using the same SCoTS procedure, with sub-trajectory lengths set to 4. Additionally, to label these new trajectories with rewards, we additionally trained reward model, r: [u az), on the original offline dataset. As shown Table [T] training on SCoTS- -aggmeend data consistently improves the performance of the original Diffuser. Even without architectural or sampling strategy changes,
26

---

by stitching sub- sib-trajectories in a way that is temporally extended and exploratory, SCoTS generates novel behaviors (e.g., trajectories that travel further) that are not present in the original suboptimal dataset.
Table 11:
Performance on D4RL benchmarks. We report normalized average returns the Diffuser and Diffuser trained on SCooISS-augmente data. Results are averaged over 50 planning seeds.
Dataset
Diffuser
Diffuser + SCoTS (ours)
hopper-mediun-122
74.3+1.4
93.5+112
walker24-medium-12
79.6
S0.240.3
hopper-nedduu--replaa--WW
93. 93.6+0.4
96. 96.6
valkerzdd-madiu--rppl.yyy
70.6+1.6
78.5+1.9
Average
79.5
87.2
Evaluation on noisy and partially observable navigation. To evaluate the robustness of SCoTS on noisy and low-quality data beyond the controlled OGBench settings, we conducted additional experiments using a mobile robot navigation task in realistic, large-scale environments (a 35m x 39m corridor and 50m 1 100m building), following the setup from HRL (Lee al. 2023a We simulated low-quality data by generating short, disconnected segments. Specifically, trajectories were collected by randomly selecting start and goal locations within short ranges (5m) using HRL (Lee al. 2023a). To handle the noisy raw 2D LiDAR measurements (512 rays over 360-degree), we converted these scans into 64x64 occupaney grid images. These images were then encoded using a pre-trainne trained 3-VAB (Higgins et al. (017) into a compact 8-dimensional latent representation, providing structured, low-dimensional state inputs for the agent. Subsequently, the SCoTS stitching method applied to augment this fragmented dataset. The results, shown in Table 12 clearly indicate that SCoTS effectively enhances performance, even with highly fragmented and noisy datasets, demonstrating its robustness and practical applicability to robot navigation tasks.
Table 12: Performance mobile robot navigation task. Goal-reaching success rates reported over 100 episodes for the HIQL (Park et al. 2023) trained with and without SCoTS augmentation.
Environment
HIQL HIQL + SCoTS (ours)
Building (50m x 110m)
27+7
53+4
Corridor (35m 39m)
OtD
31+5
Visualization of temporal distance-presed preserving latent representations. We train temporal
latent representations with dimension 32 across all environments. To visu-
alize these learned representations, we apply t-distributed stochastic neighbor embedding (t-SNE)
to project the 32-dimensional latent vectors onto a 2-dimensional plane, as shown in Figure g Recall from Equation 6 that we parameterize goal-conditioned value function V(s. g) following
(Park
ct al. 2024a):
V(s, g) -(gg)
(13)
which approximates the optimal goal-conditioned value function, defined as the maximum possible return (cumulative sum of rewards) for sparse-reward settings. Specifically, an agent receives reward of 0 if the l2 distance between states and is within small threshold 6,, and -1 otherwise. The embedding function é trained using a temporal-diffe difference objective inspired by implicit Q-learning (Kostrikov et al. 2022) on the offline dataset D. As illustrated in Figure 8 the learned representations efffctively capture the temporal proximity between states, resulting in latent spaces where states that are temporally close in the environment are also clustered closely in the embedding space.
Visualization of rollout execution. We visualize generated plan by the diffusion plan- ner trained on SCoTS- -uggmentd data, along with its corresponding rollout execution in the Sittaaze-Cintt-SSS
environment, as illustrated in Figure The initial image (top-left) shows the overall planned trajectory generated by the diffusion planner, with subgoals marked by green spheres. Subsequent images provide sequential snapshots from the rollout execution, demonstrating the agent actively pursuing and reaching these subgoals. This visualization highlights how effectively the generated high-level plan guides the lewelevel controller during task execution.
27

---

+
(a) States (Mediun)
(b) PolntMare-Yoduum--tt-- Stitic
(c)
(d) AadMazze-ModdumBBppp.o
LP
(e) States (Large) Pointthaze-argg--Sitch
(g) LatHeze-Large-Stittd
Antreaz--Larg--e-eeeeee Explore
TA
( States (Giant)
PointMaze- Saaaaaazee-nann---ee Giant-Stitedd
(k) AatMaaae-Gaaaae Giant-Stitch
Figure 8: Visualization of learned temporal Distance-rresevinng latent representations. The left- most column shows original states from maze environments of varying sizes (Medium, um, Large, Giant). Subsequent columns illustrate t-SNE projections of latent embeddings ((s) for corresponding OG- Bench datasets, maintaining the same color scheme for consistency. This visualization demonstrates how spatial proximity and structure in original state space are preserved and reflected in learned latent representations.
Visualization of trajectories generated by SCoTS. In Figure 10 11 and 12 we present rep- resentative examples of trajectories synthesized by our SCoTS framework across all considered environments and dataset types. Compared to the original trajectories provided in Figure 7 the SCoTS gennertedd trajectories clearly demonstrate extended coverage, illustrating the effectiveness of our method in augmenting the original offline datasets.
D Baseline Performance Sources
Performance scores reported for offline goal-conditioned reinforcement learning (GCRL) methods, including Goal-Conditioned Implicit Q-Learning (GCIQL) (Kostrikov et al.) 2022), Quasimetric RL (QRL) Wang et al. 2023). Contrastive RL (CRL) (Eysenbach et al.) (002), and Hierarchical Implicit Q-Lerning (HIQL) (Park et al. 2023), are sourced from Table 2 in Park et al. (2025). Scores for diffusion- generative planning methods explicitly designed for long-horizon generalization, including Generative Skill Chaining (GSC) (Mishra et al. 2023) and Compositional Diffuser (CD)
(Luo et al. 2025). are sourced from Tables and 2 in Luo et al. (2025).
28

---

Figure 9: Visualization of diffusion Planner rollout execution. The top left image shows the planned trajectory generated by the diffusion planner, with subgoals marked by green spheres. Subsequent images sequentially illustrate the agent progressing toward these subgoals in the SrtMaaz-Giant-Sttee environment, demonstrating effective guidance provided by the generated plan.
29

---

Figure 10: SCuTn-augmented trajectories for PointMaze Stitch datasets. For each PointMaze Stitch dataset, the leftmost column shows trajectories from the original OGBench dataset. The subsequent columns are examples of SCoTS-gennrated trajectories.
(a) PointMaze Medium-Stitae
(b) Seinttaaze-LargeSttaee Stitch
TATT AARAA
P
(c) PointMaze Giant-Stita
30

---

Figure 11: SCoTS-augmentt trajectories for AntMaze Stitch datasets. For each AntMaze Stitch dataset, the leftmost column shows trajectories from the original OGBench dataset. The subsequent columns are examples of SCoTS- SCoTS-generated trajectories.
O
(d) AntMaze SaaMMaze-Medilee Medium-Stitch
(e) AntMaze Saarge-Stitch
TAIAAA
(f) AntMaze Gaataaze-Ginnt.. Slant-Sitch
Figure 12: SCoTS-augmentee trajectories for AntMaze Explore datasets. For each AntMaze Explore dataset, the leftmost column shows trajectories from the original OGBench dataset. The subsequent columns examples of "CoTS-generated trajectories.
(g) AntMaze -eeizee-- -xpllor
(h) AntMaze Large-Expll
31


Reviews:
[{"review_id": "P89RF1NrY3", "reviewer": "Reviewer_C6Zi", "summary": "This work proposes a novel method called State-Covering Trajectory Stitching (SCoTS) for trajectory augmentation. The method first learns a temporal distance-preserving embedding that encodes states into a latent representation. Then, it randomly samples trajectory segments and selects candidate segments to concatenate based on the proposed progress score and novelty score. The resulting augmented trajectories are used to supplement the original offline dataset for training the final diffusion planner. Experimental results demonstrate that the method performs well on long-horizon tasks.", "questions": "1. In the remark paragraph of the Planning with Diffusion Models section, the authors claimed that the augmented dataset $\\mathcal{D}_\\text{aug}$ would extend trajectory coverage. However, as far as I understand, the augmented data does not generate new states but rather concatenates different segments of existing trajectories. Does this really bring substantial benefits compared to other diffusion planning methods that decompose trajectories into subgoals, such as Monte Carlo Tree Diffusion [1] or Hierarchical Diffuser [2]?\n\n2. In Section 3.2, are the candidate trajectory segments $\\tau_j$ sampled from the same original trajectories, or are they drawn across different trajectories?\n\n[1] Yoon, Jaesik, et al. Monte Carlo Tree Diffusion for System 2 Planning. Proceedings of the Forty-second International Conference on Machine Learning, 2025.\n\n[2] Chen, Chang, et al. Simple Hierarchical Planning with Diffusion. The Twelfth International Conference on Learning Representations, 2024.", "limitations": "The limitations of this work are discussed in the conclusion section.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The idea of introducing trajectory stitching to generate augmented data using both the proposed progress score and novelty score is innovative.\n\n2. Comprehensive experimental results on multiple tasks demonstrate the effectiveness of the work and prove that the evaluation is sufficient.\n\n3. Concrete details of the implementation are provided in the appendix to ensure the reproducibility of this work.\n\nWeaknesses:\n\n1. Lacking theoretical results to prove the correctness of the methods, especially in the directional and exploratory trajectory stitching part. The idea is interesting, but there is no guarantee that stitching different trajectory segments would help planning.\n\n2. Some minor issues:\n\n   - In the first paragraph of Section 3.2, there seems to be no difference between the unit vector $z$ and the vector itself in $z \\gets \\frac{z}{∥z∥}$.\n\n   - The state encoder $\\phi(s)$ should take the state as input, but in Eq. (8), $\\phi(\\mathcal{D})$ appears to take the dataset $\\mathcal{D}$ as input. The authors may want to use an alternative representation.", "was_revised": true, "final_justification": "I thank the authors for the detailed rebuttal and their prompt responses to the follow-up discussions. The authors have thoroughly discussed their contributions in comparison to prior works. Although I remain somewhat unclear about the contributions relative to other stitching-based planning methods, I find the paper solid and the method promising. I have increased my score by 1.", "rebuttal": "Thank you for the detailed review and constructive feedback on our work. We especially appreciate your insightful question comparing the benefits of our trajectory stitching approach against subgoal decomposition methods, which is particularly helpful for clarifying our contribution.  Please find our detailed responses below.\n\n- **“Does SCoTS really bring substantial benefits compared to methods like MCTD [1] and HD [2]?**”\n\nThank you for this important question. We would like to clarify the fundamental difference between our approach and prior methods such as MCTD [1] and HD [2].\n\nThe fundamental limitation of these methods is that their planning horizon is inherently constrained by the maximum length of trajectories seen during training. Consequently, they typically struggle with tasks requiring extensive stitching, such as those in the OGBench *Stitch* dataset, where individual trajectories are limited to 200 steps but successful task completion requires trajectories of 800+ steps.\n\nIn contrast, SCoTS systematically generates diverse, extended trajectories, effectively breaking the horizon limitation of existing planners. To demonstrate this empirically, we compare SCoTS with MCTD [1] and HD [2] on PointMaze environments with two dataset types:\n\n- Navigate: Contains trajectories long enough to directly solve test tasks.\n- Stitch: Contains short trajectories (max 4 cells), requiring extensive stitching.\n\n|  | **MCTD [1]** | **HD [2]** | **HD + SCoTS (ours)** |\n| --- | --- | --- | --- |\n| pointmaze-navigate-medium | 100 ± 0 | 100 ± 0 | 100 ± 0 |\n| pointmaze-navigate-large | 98 ± 6 | 87 ± 3 | 100 ± 0 |\n| pointmaze-stitch-medium | 90 ± 14 | 24 ± 3 | 100 ± 0 |\n| pointmaze-stitch-large | 20 ± 0 | 17 ± 2 | 100 ± 0 |\n\nAs shown, methods like HD and MCTD struggle significantly when extensive trajectory stitching is required. In contrast, HD trained with SCoTS-augmented data achieves consistent success, demonstrating our method's clear benefit.\n\n- **“Are the candidate trajectory segments sampled from the same original trajectories, or are they drawn across different trajectories?”**\n\nTo clarify, the candidate trajectory segments $\\boldsymbol{\\tau}_j$ sampled are drawn from across all different trajectories within the entire offline dataset $\\mathcal{D}$, not from the same original trajectory.\n\nThis is explicitly handled by our search mechanism described in Section 3.2 and Equation (8): $\\text{TopKNeighbors}(\\phi(\\text{end}(\\boldsymbol{\\tau}_{\\text{comp}})),\\phi(\\mathcal{D}),k).$\n\nHere, the search space $\\phi(\\mathcal{D})$ consists of the latent embeddings of the initial states of all trajectories in the offline dataset. Our method finds the nearest neighbors to the current endpoint $\\text{end}(\\boldsymbol{\\tau}_{\\text{comp}})$ from this global pool of candidates.\n\nThis cross-trajectory stitching is fundamental to effectiveness of SCoTS. By connecting segments from entirely different source trajectories, we can synthesize novel behavioral sequences that were never demonstrated together in the original data. This enables the generation of diverse, extended trajectories that explore previously unseen regions of the trajectory space, significantly enhancing the diffusion planner's generalization capabilities beyond the original data distribution.\n\nWe appreciate the reviewer highlighting this ambiguity and will clarify this point carefully in the final version of the paper.\n\n- **“How directional and exploratory trajectory stitching would help planning.”**\n\nThe rationale behind our stitching strategy is inspired by recent successes in unsupervised reinforcement learning [3, 4, 5], demonstrating that state-covering behaviors significantly benefit downstream tasks.\n\nOur work addresses a critical limitation in offline RL: datasets are often composed of disconnected, short-horizon, or narrow-distribution trajectories. Consequently, the strategy for how to stitch these trajectories becomes a crucial factor. For example, a naive stitching approach, such as randomly connecting segments with nearby endpoints, would inevitably generate cycles or static behaviors confined to local regions, failing to provide meaningful state coverage for downstream planning.\n\nIn contrast, SCoTS generates trajectories that are both temporally extended and diverse through a principled mechanism. We achieve temporal extension by guiding the stitching process along consistent directions within a learned temporal distance-preserving latent space. Diversity is ensured by assigning a unique, randomly sampled latent direction to guide the generation of each augmented trajectory. This principled strategy produces augmented datasets containing meaningful state-covering trajectories beneficial for training both diffusion planners and general GCRL methods.\n\n- **In the first paragraph of Section 3.2, there seems to be no difference between the unit vector $\\boldsymbol{z}$ and the vector itself in $\\boldsymbol{z}\\gets \\boldsymbol{z}/\\|\\boldsymbol{z}\\|$.**\n\nThank you for pointing out this. The expression $\\boldsymbol{z}\\sim\\mathcal{N}(\\mathbf{0},\\mathbf{I})\\,\\boldsymbol{z}\\gets \\boldsymbol{z}/\\|\\boldsymbol{z}\\|$ is intended to describe a two-step procedure: 1) a vector is first sampled from a standard Gaussian distribution, and 2) this vector is then normalized to become a unit vector. To enhance clarity in the final version, we will revise the text to explicitly separate these two steps,\n\n- **The state encoder $\\phi(s)$ should take the state as input, but in Eq. (8), $\\phi(\\mathcal{D})$ appears to take the dataset $\\mathcal{D}$ as input.**\n\n**$\\phi(\\mathcal{D})$** is a concise representation for the set of latent embeddings of the initial states of all trajectories within the dataset **$\\mathcal{D}$**. This set forms the search space for candidate segments. We will clarify this shorthand in the final version to improve readability.\n\nReferences:\n\n[1] Yoon et al., Monte Carlo Tree Diffusion for System 2 Planning. ICML, 2025.\n\n[2] Chen et al., Simple Hierarchical Planning with Diffusion. ICLR, 2024.\n\n[3] Park et al., Lipschitz-constrained Unsupervised Skill Discovery. ICLR, 2022.\n\n[4] Park et al., METRA: Scalable Unsupervised RL with Metric-Aware Abstraction. ICLR, 2024.\n\n[5] Bae et al., TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations. CoRL, 2024."}, {"review_id": "8AuL9RqcyI", "reviewer": "Reviewer_RD6p", "summary": "This paper introduces State-Covering Trajectory Stitching (SCOTS), a method to improve diffusion-based planners that are fundamentally limited by the diversity and horizon of their offline training data. SCOTS addresses this by augmenting datasets with diverse, long-horizon trajectories. The framework first learns a temporal distance-preserving latent representation of the environment's states. It then iteratively stitches short trajectory segments together, selecting segments that balance progress along a random exploratory direction with novelty to effectively expand state-space coverage. A diffusion-based model refines the connection points between segments to ensure the final trajectories are dynamically consistent. Experiments demonstrate that training on SCOTS-augmented data significantly enhances the generalization and long-horizon planning capabilities of diffusion planners. The method also provides a notable performance boost to widely-used offline goal-conditioned reinforcement learning algorithms.", "questions": "1. Figure 1 in the paper demonstrates that the current Diffuser method fails to generalize well to these out-of-distribution tasks. However, Figure 3.b in the original Diffuser paper (https://arxiv.org/pdf/2205.09991) seems to show that it can achieve a similar trajectory stitching effect to the proposed method. How do the authors explain this apparent contradiction?\n\n2. Why is it necessary to balance the randomly sampled latent direction with the novelty relative to previously visited states?", "limitations": "See the first point in the Weaknesses section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. This work demonstrates significant performance improvements over the compared baselines on the OGBench benchmark.\n\nWeaknesses\n\n1. The paper emphasizes that the proposed method is *reward-free* as a key distinction from prior works. However, there is no analysis or discussion throughout the paper to explain the importance or effectiveness of this reward-free property. In fact, if I understand correctly, if the augmented data does not contain rewards, the method would not be effective for most offline RL tasks and would only be applicable to specific goal-conditioned tasks, which further limits the applicability of this approach.\n\n2. The writing is sometimes confusing, and several claims or proposed solutions lack justification—for example, through illustrative toy experiments or empirical evidence on real tasks. For instance: why is it necessary to balance the randomly sampled latent direction with novelty relative to previously visited states? Aren’t both intended to enhance exploration?\n\n3. The proposed method incorporates many components from prior works and several technical mechanisms, which leans toward an engineering-heavy design—which is fine. However, the paper does not sufficiently demonstrate, even through ablation studies, the necessity of including these components. I suggest adding experiments to support this aspect.", "was_revised": true, "final_justification": "The authors provided a thoughtful and comprehensive rebuttal that effectively addressed all my initial concerns. Their clarifications resolved key questions about the method's design and motivation, and the new experiments — particularly on offline RL benchmarks and through ablation studies — provide strong empirical support.\n\nConcerns raised by other reviewers, such as computational cost, were also addressed with reasonable explanations. While the original experimental scope was limited, the additional results broaden the applicability and reinforce the contributions.\n\nBased on these updates, I have increased my score and recommend acceptance. I encourage the authors to include the new results in the final version to further strengthen the paper.", "rebuttal": "Thank you for the detailed review and constructive feedback on our work. We especially appreciate your insightful question about the necessity of balancing directional progress with exploratory novelty in our trajectory stitching method, which is particularly helpful for clarifying our contribution. Please find our detailed responses below.\n\n- **“Contradiction between our Fig.1 and Diffuser original paper's Fig.3.b?”**\n\nThanks for asking this valid question. The success shown in the original Diffuser paper's Figure 3.b was on a relatively simple toy experiments. In such settings, standard U-Net-based diffusion models can perform well. However, this architectural property becomes a critical limitation in the long-horizon and high-dimensional OOD settings we target. We hypothesize this is because the aggressive downsampling from pooling and strided convolutions reduces the bottleneck representation so drastically that even small kernels can access distant sequence information, a property that is helpful in simple settings but detrimental in complex ones where temporal details are lost.\n\nThis limitation has been recognized issue in the community, as acknowledged in prior works [2, 3]. For example, the appendix of [3] presents a similar analysis and empirically illustrates this failure mode (Figure 4 in Appendix). Therefore, we argue there is no contradiction. The original work demonstrated success in a constrained setting, while our work and Figure 1 address the failure of these models to generalize to more challenging scenarios.\n\n- **“Why balance directional progress with novelty?”**\n\nAs illustrated in our paper's Figure 4, using only the progress score ($\\beta=0$), which quantifies the alignment with a latent exploration direction, is effective at generating long, temporally-extended trajectories with clear directional distinctions.\n\nHowever, relying solely on this directional guidance can be suboptimal. The learned temporal distance-preserving embedding $\\phi$ is a powerful but imperfect approximation of the true environment topology. Due to embedding errors, the geometry of the latent space can be distorted. Consequently, a straight line between two latent states, $\\phi(s)$ and $\\phi(s')$, may not correspond to a feasible path in the underlying MDP. Therefore, exploratory detours are often necessary to find a valid temporally extended path.\n\nThis is precisely where the novelty score becomes critical. It encourages these exploratory detours by rewarding the selection of segments that lead to less-visited states. This allows the agent to navigate around the imperfections in the latent space, discovering more temporally extended paths.\n\nTo demonstrate this complementary effect quantitatively, we conducted an additional ablation study on 2 OGBench tasks. The table below compares the performance of the HD planner trained with SCoTS data generated using different novelty weights $\\beta$ (5 seeds).\n\n|  | **HD** | **beta=0** | **beta=1** | **beta=2** | **beta=4** | **beta=8** |\n| --- | --- | --- | --- | --- | --- | --- |\n| antmaze-stitch-large | 36 ± 2 | 87 ± 2 | 94 ± 1 | 93 ± 1 | 92 ± 1 | 85 ± 3 |\n| antmaze-stitch-giant | 0 ± 0 | 63 ± 3 | 72 ± 2 | 87 ± 2 | 89± 2 | 74 ± 3 |\n\nThe results suggest that while directional stitching alone ($\\beta=0$) provides a notable performance improvement, introducing and balancing it with the novelty score ($\\beta>0$) yields substantial further gains. This effect is especially pronounced on the more complex Giant task. \n\n- **Applicability of SCoTS to general offline RL.**\n\nWhile our paper focuses on tasks where extrinsic rewards are absent, the SCoTS framework is not limited to goal-conditioned tasks. \n\nTo clarify, reward-free refers to our stitching mechanism, which is guided by the intrinsic temporal structure rather than extrinsic task rewards. This allows the core data augmentation to be task-agnostic. To demonstrate its applicability to general, dense-reward offline RL, we conducted additional experiments on four standard MuJoCo locomotion benchmarks from D4RL, utilizing suboptimal medium and medium-replay datasets.\n\nFor a fair comparison, we used the same Diffuser architecture and planning horizon (32) as described in the original paper. We upsample the original data to 2M samples using the same SCoTS procedure, with sub-trajectory lengths set to 4. Additionally, to label these new trajectories with rewards, we additionally trained a reward model, $r_t = f_{\\omega}(s_t,a_t)$, on the original offline dataset. The table below shows the normalized average returns over 50 planning seeds.\n\n|  | **Diffuser** | **Diffuser + SCoTS (ours)** |\n| --- | --- | --- |\n| hopper-medium | 74.3 ± 1.4 | 93.5 ± 1.21 |\n| walker2d-medium | 79.6 ± 0.55 | 80.2 ± 0.5 |\n| hopper-medium-replay | 93.6 ± 0.4 | 96.7 ± 0.62 |\n| walker2d-medium-replay | 70.6 ± 1.6 | 78.5 ± 1.86 |\n| **average** | 79.5 | **87.2** |\n\nThe results show that training on SCoTS-augmented data significantly and consistently improves the performance of the original Diffuser. Even without architectural or sampling strategy changes, by stitching sub-trajectories in a way that is temporally extended and exploratory, SCoTS generates novel behaviors (e.g., trajectories that travel further) that are not present in the original suboptimal dataset.\n\n- **“Additional experiments to demonstrate the necessity of each component.”**\n\nWe appreciate the reviewer's valuable suggestion. We have conducted additional ablation studies to empirically demonstrate the necessity of each key component of SCoTS.\n\nTo isolate the contribution of our learned latent space, we compare the performance of the HD planner trained on SCoTS-augmented data where the stitching process was guided by either our temporal distance-preserving embedding or raw state space.\n\n|  | **HD + SCoTS (w/o temporal embedding)** | **HD + SCoTS (w/ temporal embedding)** |\n| --- | --- | --- |\n| pointmaze-stitch-large | 93 ± 0 | 100 ± 0 |\n| pointmaze-stitch-giant | 52 ± 1 | 100 ± 0 |\n| antmaze-stitch-large | 45 ± 1 | 93 ± 1 |\n| antmaze-stitch-giant | 7 ± 2 | 87 ± 2 |\n\nThe results clearly show that the temporal distance-preserving representation is crucial for effective stitching. This is especially true in high-dimensional environments like antmaze, where a small Euclidean distance in the raw state space (e.g., between two similar joint configurations) does not guarantee reachability. Even in pointmaze, where state-space distance is more intuitive, the learned compact latent space provides a better-structured representation for stitching temporally extended trajectories.\n\nTo demonstrate the importance of the refinement step, we compare the performance of both a diffusion planner (HD) and a GCRL algorithm (HIQL) trained on SCoTS data generated with and without the diffusion-based refinement.\n\n|  | **HD + SCoTS (w/o refinement)** | **HD + SCoTS (w/ refinement)** | **HIQL + SCoTS (w/o refinement)** | **HIQL + SCoTS (w/ refinement)** |\n| --- | --- | --- | --- | --- |\n| antmaze-stitch-large | 85 ± 3 | 93 ± 1 | 52 ± 2 | 91 ± 2 |\n| antmaze-stitch-giant | 53 ± 1 | 87 ± 2 | 11 ± 2 | 55 ± 5 |\n\nThese results reveal the critical importance of the refinement component. Without refinement, the connection points between stitched segments can suffer from large dynamic inconsistency errors, as illustrated in Figure 7 of our appendix. This performance degradation is particularly significant when training GCRL algorithms like HIQL, which are highly sensitive to the dynamic validity of the training transitions. \n\nReferences:\n\n[1] Janner et al., Planning with Diffusion for Flexible Behavior Synthesis. ICML, 2022.\n\n[2] Chen et al., Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. NeurIPS, 2024.\n\n[3] Chen et al., Simple Hierarchical Planning with Diffusion. ICLR, 2024."}, {"review_id": "NpoXcmKMbs", "reviewer": "Reviewer_nESC", "summary": "This paper proposed a novel, reward-free method for trajectory stitching in offline long-horizon planning. Unlike previous method, this method performs planning in a temporal distance-preserving embedding space, and encourages the novelty of sub-trajectory generation to make the exploration of whole state spaces. Empirical results are provided to support the claim for the proposed method.", "questions": "see weakness", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**strength:** The paper is well-structed and easy to follow. The idea is well-motivated and the empirical analysis is sufficient to demonstrate the effectiveness of proposed method.\n\n**weakness:**\n\n1. The learning of Temporal Distance-Preserving Embedding. One major concern for this part is the convergence and optimality of the learning of this embedding. That is, by the loss function provided by Eq.(7), what kind of embedding function $\\phi$ will be outputted? Can it actually learns the claimed embedding of optimal temporal distance $d^*(s,g)$? More evidences (better from theoretical aspect) should be provided.\n\n2. (Bias and Generalization ability) Following weakness 1, if the learned embedding is biased, how will it affect the subsequent process and final performance? Even if we can successfully find the neighbor states by Eq.(8), how can it guarantee that the inverse dynamics model is able to successfully predict the action to combine the two states (end($\\tau_{comp}$), initial($\\tau_{best}$)), if this state pair is never demonstrated in the dataset.\n\n3. Some details are not clear. For instance, the formulation of TopKNeighbours() function in Eq.(8) is not given; Section 3.3 is not well-structured, where the stitcher should be more detailed formulated and how can it avoid to 'exhibit minor dynamic inconsistencies or sub-optimal transitions'? Furthermore, all the offline RL agents should be guided by reward, so how does reward affect the agent in this method?\n\n4. Ablation study and sensitive analysis are not sufficient. There should be many hyperparameters in this method that will significantly affect the performance, i.e., the length of sub-trajectories, the K in Eq.(8) and (10), and the $\\beta$ in Eq.(11). However, their sensitivities are not well analyzed.\n\n5. (Optional) We are happy to see the results on other offline RL benchmarks such as D4RL (e.g., MuJoCo), to demonstrate the effectiveness of the proposed method on a broader scope.\n\n**Additional comments:** In my opinion, this work is potential. Although a borderline accept is given currently, I am open for further discussion and eager to see the improvements during rebuttal.", "was_revised": true, "final_justification": "All my concerns in the review have been well addressed by the authors, so I suggest the acceptance for this paper.", "rebuttal": "Thank you for the detailed review and constructive feedback on our work. We especially appreciate your helpful comments and suggestions for further improving the work. Please find our detailed responses below.\n\n- **“Can $\\phi$ actually learns the embedding of optimal temporal distance?”**\n\nLearning temporal embedding $\\phi$ is grounded in the well-established equivalence between the optimal temporal distance and the optimal goal-conditioned value function [1, 2, 3]: $V^{\\*}(\\boldsymbol{s},\\boldsymbol{g}) = -d^{\\*}(\\boldsymbol{s},\\boldsymbol{g})$\n\nHere, $V^{\\*}(\\boldsymbol{s},\\boldsymbol{g})$ is the maximum possible cumulative sum of rewards for setting where an agent receives a reward of $0$ if the $l_2$ distance between states $\\boldsymbol{s}$ and $\\boldsymbol{g}$  is within a small threshold $\\delta_g$, and $-1$ otherwise. We leverage this by parameterizing our value function directly as a distance in the latent space: $V(\\boldsymbol{s}, \\boldsymbol{g}) \\coloneqq -||\\phi(\\boldsymbol{s}) - \\phi(\\boldsymbol{g})||_2$\n\nBy training $\\phi$ to minimize a standard IQL-based temporal difference (TD) error for this value function, we make $V(\\boldsymbol{s}, \\boldsymbol{g})$  to approximate the true optimal value $V^{\\*}(\\boldsymbol{s}, \\boldsymbol{g})$ . Consequently, this implicitly forces our learned latent distance, $||\\phi(\\boldsymbol{s}) - \\phi(\\boldsymbol{g})||_2$ to approximate the optimal temporal distance, $d^{\\*}(\\boldsymbol{s},\\boldsymbol{g})$. Empirically, this approximation proves effective, though we acknowledge it remains an approximation.\n\n- **“How will embedding error affect the subsequent process and final performance?”**\n\nThank you for this important question. The robustness of our method to imperfections in the learned embedding is a core design consideration. We explicitly acknowledge that the embedding $\\phi$ can be a biased approximation, and SCoTS incorporates two key mechanisms to mitigate the potential negative effects during the stitching process.\n\nFirst, embedding errors can lead to the retrieval of candidate segments that are not dynamically consistent with the current trajectory. This can result in stitches with large dynamic inconsistency errors at the connection points. To address this, we employ our Diffusion-based Stitching Refinement step, which is specifically designed to smooth the transition between segments and ensure dynamic feasibility, as shown in Figure 7 of our Appendix.\n\nTo demonstrate its necessity, we ablate this component. The results show that removing refinement significantly degrades performance for both the diffusion planner (HD) and, more dramatically, the GCRL algorithm (HIQL), which is highly sensitive to the dynamic validity of transitions.\n\n|  | **HD + SCoTS (w/o refinement)** | **HD + SCoTS (w/ refinement)** | **HIQL + SCoTS (w/o refinement)** | **HIQL + SCoTS (w/ refinement)** |\n| --- | --- | --- | --- | --- |\n| antmaze-stitch-large | 85 ± 3 | 93 ± 1 | 52 ± 2 | 91 ± 2 |\n| antmaze-stitch-giant | 53 ± 1 | 87 ± 2 | 11 ± 2 | 55 ± 5 |\n\nSecond, geometric distortions in the latent space can make a straight-line exploration path suboptimal. Consequently, a straight line between two latent states, $\\phi(s)$ and $\\phi(s')$, may not correspond to a feasible path in the underlying MDP. Therefore, exploratory detours are often necessary to find a valid temporally extended path. This is precisely where the novelty score becomes critical. It encourages these exploratory detours by rewarding the selection of segments that lead to less-visited states. This allows the agent to navigate around the imperfections in the latent space, discovering more temporally extended paths.\n\nThe following ablation study on the novelty weight $\\beta$ demonstrates this complementary effect. The results show that while directional stitching alone $(\\beta=0)$ significantly outperforms the baseline, introducing a novelty score $(\\beta>0)$ provides substantial further gains.\n\n|  | **HD** | **beta=0** | **beta=1** | **beta=2** | **beta=4** | **beta=8** |\n| --- | --- | --- | --- | --- | --- | --- |\n| antmaze-stitch-large | 36 ± 2 | 87 ± 2 | 94 ± 1 | 93 ± 1 | 92 ± 1 | 85 ± 3 |\n| antmaze-stitch-giant | 0 ± 0 | 63 ± 3 | 72 ± 2 | 87 ± 2 | 89± 2 | 74 ± 3 |\n- **“Formulation of TopKNeighbours”**\n\nThe function $\\text{TopKNeighbors}(\\text{query}, \\text{search space}, k)$ is a standard k-nearest neighbor search algorithm. In the context of our paper, its query is the latent embedding of the final state of the currently composed trajectory, $\\phi(\\text{end}(\\boldsymbol{\\tau}_{\\text{comp}}))$, and the search space is the the set of latent embeddings of the initial states of all trajectories within the entire offline dataset, denoted by the shorthand $\\phi(\\mathcal{D})$. \n\nThe function operates by searching this space to find the $k$ vectors closest to the query, using the Euclidean distance in the learned latent space as the metric. It then returns the $k$ original trajectory segments from the dataset that correspond to these nearest initial states, which form the candidate set for the next stitching step. (Practical implementation can be found in our Appendix B.) \n\nWe appreciate the reviewer highlighting this ambiguity and will clarify this point carefully in the final version of the paper.\n\n- **“How can diffusion-based stitching refinement avoid to “exhibit minor dynamic inconsistencies or sub-optimal transitions”?**\n\nThe key to avoiding inconsistencies lies in the fact that our method is generative, not simple concatenation. A naive approach would be to simply concatenate two segments, which could create an abrupt, or even dynamically impossible transition at the stitching point. Our diffusion-based stitcher avoids this by generating an entirely new bridging path. Because the stitcher is a diffusion model trained on the entire dataset of real trajectories, it learns a strong prior over the distribution of physically plausible state sequences.  When conditioned on the boundary states ($\\boldsymbol{s}_1$ and  $\\boldsymbol{s}_H$), it samples a trajectory from the learned distribution of valid trajectories.\n\n- **“Ablation study and sensitivity analysis.”**\n\nThank you for your valuable feedback. Following the suggestion, we have conducted additional ablation and sensitivity analyses to better understand the impact of key hyperparameters and the necessity of our core components.\n\n**1. Sensitivity to Sub-trajectory Length $H$**\n\n|  | **H=26** | **H=52** | **H=104** |\n| --- | --- | --- | --- |\n| antmaze-stitch-giant | 87 ± 2 | 89 ± 2 | 87 ± 2 |\n| antmaze-explore-large | 98 ± 1 | 93 ± 1 | 88 ± 2 |\n\nThe results show that on the stitch dataset, performance is relatively robust to the choice of $H$. However, on the explore dataset, shorter segments perform best. We hypothesize this is because the explore dataset consists of low-quality, noisy trajectories. Using shorter segments allows SCoTS to be more selective, finding and connecting the temporally-extended parts of trajectories.\n\n**2. Sensitivity to Number of Retrieved Segments $K$**\n\n|  | **K=3** | **K=10** | **K=20** |\n| --- | --- | --- | --- |\n| antmaze-stitch-giant | 65 ± 3 | 87 ± 2 | 89 ± 2 |\n| antmaze-explore-large | 72 ± 2 | 98 ± 1 | 97 ± 1 |\n\nPerformance is robust as long as $K$ is not too small. A very small $K$ limits the diversity of candidate segments, hindering the effectiveness of our progress and novelty-based selection. While performance is stable for larger $K$, we expect an excessively large $K$ could eventually degrade performance by increasing the chance of retrieving dynamically inconsistent segments.\n\n**3. Sensitivity to Novelty Weight $\\beta$**\n\nWe respectfully refer the reviewer to our detailed response to second question.\n\n**4. Necessity of Core Components**\n\nThey are discussed in the rebuttal for reviewer RD6p due to the space limitation.\n\n**“We are happy to see the results on other offline RL benchmarks”** and **“How does reward affect the agent in this method?”**\n\nWhile our paper focuses on tasks where extrinsic rewards are absent, the SCoTS framework is not limited to goal-conditioned tasks. \n\nTo clarify, reward-free refers to our stitching mechanism, which is guided by the intrinsic temporal structure rather than extrinsic task rewards. This allows the core data augmentation to be task-agnostic. To demonstrate its applicability to general, dense-reward offline RL, we conducted additional experiments on four standard MuJoCo locomotion benchmarks from D4RL, utilizing suboptimal medium and medium-replay datasets.\n\nFor a fair comparison, we used the same Diffuser architecture and planning horizon (32) as described in the original paper. We upsample the original data to 2M samples using the same SCoTS procedure, with sub-trajectory lengths set to 4. Additionally, to label these new trajectories with rewards, we additionally trained a reward model, $r_t = f_{\\omega}(s_t,a_t)$, on the original offline dataset. The table below shows the normalized average returns over 50 planning seeds.\n\n|  | **Diffuser** | **Diffuser + SCoTS (ours)** |\n| --- | --- | --- |\n| hopper-medium | 74.3 ± 1.4 | 93.5 ± 1.21 |\n| walker2d-medium | 79.6 ± 0.55 | 80.2 ± 0.5 |\n| hopper-medium-replay | 93.6 ± 0.4 | 96.7 ± 0.62 |\n| walker2d-medium-replay | 70.6 ± 1.6 | 78.5 ± 1.86 |\n| **average** | 79.5 | **87.2** |\n\nThe results show that training on SCoTS-augmented data consistently improves the performance of the original Diffuser. Even without architectural or sampling strategy changes, by stitching sub-trajectories in a way that is temporally extended and exploratory, SCoTS generates novel behaviors (e.g., trajectories that travel further) that are not present in the original suboptimal dataset.\n\nReferences:\n\n[1] Kaelbling et al., . Learning to achieve goals. IJCAI, 1993.\n\n[2] Wang et al., Optimal goal-reaching reinforcement learning via quasimetric learning. ICML, 2023.\n\n[3] Park et al., Foundation Policies with Hilbert Representations. ICML, 2024."}, {"review_id": "alkU7OANcb", "reviewer": "Reviewer_tqRC", "summary": "The authors propose State-Covering Trajectory Stitching (SCoTS), a reward-free augmentation framework that generates long and diverse trajectories by incrementally stitching short segments. The method works in three stages: (1) learning a temporal-distance-preserving latent embedding to measure trajectory connectivity, (2) iteratively selecting segments based on directional progress and novelty in latent space, and (3) refining transitions between segments using a diffusion model to ensure dynamic consistency. The contribution lies in the combination of directional latent space exploration with a diffusion-based refinement pipeline for reward-free stitching.\n\nThe method is evaluated on the OGBench benchmark using PointMaze and AntMaze environments. Metrics include task success rate across various maze sizes and difficulty levels. Comparisons are made with multiple baselines, including GCIQL, HIQL, CRL, CD, and GSC, under both standard and augmented training conditions.", "questions": "1. How robust is SCoTS to noisy or low-quality datasets beyond the controlled OGBench settings, e.g., occupancy maps?\n2. Can SCoTS be extended or adapted for online learning or continual RL settings, where the state space evolves, e.g., dynamics objects?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- **Data Augmentation Method:** SCoTS introduces a trajectory-level augmentation method tailored to diffusion planners. It cleverly combines temporal-distance-preserving embeddings, novelty-driven segment selection, and diffusion-based refinement.\n- **Utility:** SCoTS-augmented data not only improves diffusion planners but also significantly boosts the performance of traditional offline goal-conditioned RL methods.\n- **Reproducibility:** The paper is transparent about computational costs and issues with temporal embeddings in asymmetric or disconnected environments.\n\n**Weaknesses**\n- **Computational Overhead:** The three-stage pipeline, especially the diffusion-based refinement step, is computationally intensive. This may limit applicability in real-time or resource-constrained settings.\n- **Assumption of Symmetric Temporal Embeddings:** The embedding method used may struggle in environments with highly asymmetric dynamics or irreversible transitions.\n- **Limited Real-World Evaluation:** All experiments are on synthetic benchmarks. It remains unclear how well the method scales or adapts to high-dimensional real-world tasks, especially those with complex sensory inputs.\n- **Evaluation Bias Toward Diffusion Planners:** While traditional GCRL methods are evaluated, the core of the paper’s benchmarks and claims primarily cater to improving diffusion planners.\n- **Blending of Task-Objectives**: Given the nature of navigation robotics tasks, SCoTS could benefit from blending diffusion and gradients from task objectives as in [1], improving collision-free performance.\n\n[1] Carvalho, Joao, et al. \"Motion planning diffusion: Learning and planning of robot motions with diffusion models.\" 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023.", "was_revised": true, "final_justification": "The authors have addressed all my concerns about the method's applicability in further settings. Hence, I am leaning towards accepting this paper.", "rebuttal": "Thank you for the detailed review and constructive feedback on our work. We especially appreciate your insightful question regarding the generalizability of SCoTS beyond the controlled OGBench setting. Please find our detailed responses below.\n\n- **“How robust is SCoTS to noisy or low-quality datasets beyond the controlled OGBench settings, e.g., occupancy maps?”**\n\nTo evaluate the robustness of SCoTS on noisy and low-quality data beyond the controlled OGBench settings, we conducted additional experiments using a mobile robot navigation task in realistic, large-scale environments (a 35m x 39m corridor and a 50m x 110m building), following the setup from HRL [1].\n\nWe simulated low-quality data by generating short, disconnected segments. Specifically, trajectories were collected by randomly selecting start and goal locations within short ranges (5m) using HRL [1]. To handle the noisy raw 2D LiDAR measurements (512 rays over 360°), we converted these scans into 64x64 occupancy grid images. These images were then encoded using a pre-trained $\\beta$-VAE into a compact 8-dimensional latent representation, providing structured, low-dimensional state inputs for the agent. Subsequently, the SCoTS stitching method was applied to augment this fragmented dataset.\n\nThe table below shows the goal-reaching success rate (over 100 episodes) of HIQL when trained with and without our SCoTS data augmentation.\n\n|  | **HIQL** | **HIQL + SCoTS (ours)** |\n| --- | --- | --- |\n| Building | 27 ± 7 | 53 ± 4 |\n| Corridor | 0 ± 0 | 31 ± 5 |\n\nThese results clearly indicate that SCoTS effectively enhances performance, even with highly fragmented and noisy datasets, demonstrating its robustness and practical applicability to robot navigation tasks.\n\n- **“Can SCoTS be extended or adapted for online learning or continual RL settings?”**\n\nAlthough our work primarily addresses the offline setting, the SCoTS framework can naturally be extended to online or continual RL settings. One straightforward approach would involve periodically incorporating new trajectories collected by the agent into the training dataset. This evolving dataset could then be used to periodically retrain both the temporal distance-preserving embedding $\\phi$ and the diffusion-based stitcher $p_{\\theta}^{\\text{stitcher}}$. This strategy aligns with successful online data augmentation approaches demonstrated in prior work [2]. We believe further extending the idea behind SCoTS to other setting is an exciting future research direction.\n\n- “**Computational overhead”**\n\nWe acknowledge that SCoTS introduces additional computational overhead, primarily due to the training of the diffusion-based stitcher and the trajectory augmentation process, which together roughly equate to 50% of the training time required for the final diffusion planner.\n\nHowever, we believe this overhead is justified by substantial performance gains. For instance, on challenging tasks such as antmaze-explore-giant, SCoTS improved the hierarchical diffusion planner's success rate by up to seven times. Such considerable improvement in generalization, achieved through a single offline augmentation phase, is highly beneficial, particularly when compared to the significantly higher costs associated with collecting a diverse set of high-quality, long-horizon data.\n\n- “**Assumption of Symmetric Temporal Embeddings”**\n\nAs we discussed in our paper's limitations section, our temporal distance-preserving embedding assumes symmetric temporal embeddings. We believe this limitation can be resolved by leveraging universal quasimetric embedding as explored in [3] which allows the representation to more faithfully model irreversible dynamics, which we leave for future work.\n\n- **“Evaluation bias toward diffusion planners”**\n\nWhile our motivation and main narrative center around enhancing diffusion planners, we emphasize that our experimental results show substantial improvements for traditional offline goal-conditioned RL (GCRL) algorithms as well.\n\nSpecifically, our work is the first, to our knowledge, to enable GCRL methods to surpass 50% success rates on particularly challenging OGBench tasks (antmaze-stitch-giant and antmaze-explore-large). These results demonstrate that SCoTS serves as a general-purpose data augmentation framework that significantly improves performance across diverse offline RL algorithms, not merely diffusion-based approaches. \n\nWe believe these results strengthen rather than weaken our contribution.\n\n- “**Given the nature of navigation robotics tasks, SCoTS could benefit from blending diffusion and gradients from task objectives as in [4], improving collision-free performance.**”\n\nWe thank the reviewer’s suggestion. We would like to clarify that our SCoTS framework and the suggested guidance mechanism are orthogonal and complementary. SCoTS is a data augmentation method that enriches the training data to provide the planner with a stronger, more diverse prior. In contrast, the guidance technique is an inference-time method that refines a generated plan to satisfy specific constraints like collision avoidance. We believe this combination is an interesting direction for future research and will add a discussion of this potential synergy in the final version of the paper.\n\nReferences:\n\n[1] Lee et al., Adaptive and Explainable Deployment of Navigation Skills via Hierarchical Deep Reinforcement Learning. ICRA, 2023.\n\n[2] Lu et al., Synthetic Experience Replay. NeurIPS, 2023.\n\n[3] Wang et al., Optimal goal-reaching reinforcement learning via quasimetric learning. ICML, 2023.\n\n[4]  Carvalho et al., Motion planning diffusion: Learning and planning of robot motions with diffusion models. IROS, 2023."}]

Author rebuttal:

Meta-review: The paper introduces a novel technique for trajectory augmentation to be used in conjunction with trajectory diffusion methods to generate a more diverse and covering dataset that can be used in offline RL algorithms (e.g., goal-condition RL or diffusion planner). The paper illustrates the effectiveness of the proposed method in a few OGBench domains and show a non-trivial improvement across offline RL algorithms and initial datasets.

Trajectory datasets tend to provide a limited level of coverage as they are mostly composed by "expert" demonstrations and any "downstream" method  (BC/offline RL/diffusion-based planners) suffer from this limitations in terms of the breadth of problems that can be effectively solved. Finding ways to reliably expand the coverage and diversity of the initial dataset (e.g., by stitching trajectories) has the potential of directly improving the performance of many different RL algorithms. As such, there was clear consensus about the significance of the problem studied in this paper. Nonetheless, in the first round of reviews, the reviewers raised concerns on the proposed method and the empirical validation. After the rebuttal, all reviewers agreed on the fact that the clarifications provided by the authors and the additional empirical evidence on the effectiveness of the methods across other domains successfully resolved their initial concerns. As a result, there is now a general agreement towards accepting the paper.

I encourage the authors to improve their initial submission on these aspects:
* Integrate the new results: the precise implementation of the proposed method in the initial experiments may suggest that its effectiveness was tightly connected with the specific domain (Ant/PointMaze navigation). Showing that it is more broadly applicable and effective was very important to support the generality of the method.
* Include the discussion on the additional references suggested the authors.
* As the proposed method includes several components, it is crucial to include the ablations from the rebuttal to provide more support to how and why each aspect of the algorithm contributes to the final results.
Decision: Accept (poster)