Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference
Authors: Benjamin Eysenbach
Vivek Myers
Russ Salakhutdinov
Sergey Levine
Venue: NeurIPS 2024 poster (2024)
Keywords: contrastive learning
prediction
planning
inference
time-series
Primary area: probabilistic_methods
TL;DR: While prediction and planning over time series data is challenging, these problems have closed form solutions in terms of temporal contrastive representations

Abstract: Given time series data, how can we answer questions like ``what will happen in the future?'' and ``how did we get here?'' These sorts of probabilistic inference questions are challenging when observations are high-dimensional. In this paper, we show how these questions can have compact, closed form solutions in terms of learned representations. The key idea is to apply a variant of contrastive learning to time series data. Prior work already shows that the representations learned by contrastive learning encode a probability ratio. By extending prior work to show that the marginal distribution over representations is Gaussian, we can then prove that joint distribution of representations is also Gaussian. Taken together, these results show that representations learned via temporal contrastive learning follow a Gauss-Markov chain, a graphical model where inference (e.g., prediction, planning) over representations corresponds to inverting a low-dimensional matrix. In one special case, inferring intermediate representations will be equivalent to interpolating between the learned representations. We validate our theory using numerical simulations on tasks up to 46-dimensions.

Inference via Interpolation:
Contrastive Representations Provably Enable Planning and Inference
Benjamin Eysenbach
Vivek Myers* UC Berkeley vmyers@berkeley. edu
Princeton University eysebbach@princeton..cm edu
Ruslan Salakhutdinov
Sergey Levine
Carnegie Mellon University rsalakhu8cs, cmu, edu
UC Berkeley svlevine@eecs. berkeley. edu
Abstract
Given time series data, how we questions like "what will happen in the future?" and "how did we get here?" These sorts of probabilistic inference this paper,
questions are challenging when observations are high-dimensional. we show how these questions can have compact, closed form solutions in terms of learned representations. The key idea apply variant of contrastive learning to time series data. Prior work already shows that the representations learned by contrastive learning encode probability ratio. By extending prior work show that the marginal distribution over representations is Gaussian, we can then prove that joint distribution representations also Gaussian. Taken together, these results show that representations learned via temporal contrastive learning follow Gauss-Markov chain, a graphical model where inference (e-g-. prediction, planning) over representations corresponds to inverting low-dimensional matrix. In one special case, inferring intermediate representations will be equivalent to interpolating between the learned representations, We validate our theory using numerical simulations on tasks up to 46-dimensions.
Introduction
Probabilistic modeling of time-series data has applications ranging from robotic control [1] material science [2], from cell biology [3] to astrophysics [4]. These applications are often concerned with two questions: predicting future states (e.g., what will this cell look like in an hour), and inferring trajectories between two given states. However, answering these questions often requires reasoning over nigh-dimensional data, which can be challenging most tools in the standard probabilistic toolkit require generation. Might it be possible to use discriminative methods (e.g., contrastive learning) to perform such inferences?
Many prior works aim to learn representations that are easy to predict while retaining salient bits of information. For time-series data, want the representation to remain sufficient statistic for distributions related to time for example, they should retain bits required to predict future states (or representations thereof). While generative methods [5-8] have this property, they tend to be computationally expensive (see, e.g., (9]) and can be challenging to scale to high-dimensional observations.
'Equal contribution.
Cooe: attps://githut com/vvivelmyyarr/ccttaatttii
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

A
Latent Space
small don
easy planning
26
O Observation Space
large dim
hard planning
Figure I: We apply temporal contrastive learning to observation pairs to obtain representations (b(2)), (x(++x)) such that (v))))) is close to (T(++)). While inferring waypoints in the high- dimensional observation space is challenging, we show that the distribution over intermediate latent representations has closed form solution corresponding to linear interpolation between the initial and final representations.
In this paper, we will study how contrastive methods (which are discriminative, rather than generative) can perform inference over times series. Ideally, we want representations of observations æ to be sufficient statistic for temporal relationships (e.g., does x' occur after x??) but need not retain other information about x (e-g- the location of static objects). This intuition motivates us study how contrastive representation learning methods [10-14] might be used to solve prediction and planning problems on time series data. While prior works in computer vision [10, 15] and natural language processing (NLP) [16] often study the geometry of learned representations, our results show how geometric operations such as interpolation are related to inference. Our analysis will focus on regularized version of the symmetrized infoNCE objective [17]. generating positive examples by sampling pairs of observations from the same time series data. We will study how representations learned in this way can facilitate two inference questions: prediction and planning. As a stepping stone, we will build upon prior work [20] to show that regularized contrastive learning should produce representations whose marginal distribution is an isotropic Gaussian distribution.
The main contribution of this paper is demonstrate how intermediate and future time steps in time series can be inferred easily using contrastive representations. This inference problem captures a number of practical tasks: interpolation, in-filling, and even planning and control, where the intermediate steps represent states between a stand and goal. While ordinarily these problems require an iterative inference or optimization procedure, with contrastive representations this can be done simply by inverting win-iinesionna matrix. In one special case, inference will correspond to linear interpolation. Our first step is to prove that, under certain assumptions, the distribution over future representations has Gaussian distribution, with a mean that is a linear function of the initial state representation (Lemma 1). This paves the main to our main result (Theorem 2): given an initial and final stute, we show that the posterior distribution over an intermediate state representations also follows a Gaussian distribution. Said in other words, the representations follow Gauss-Markon chain, 3 wherein any joint conditional distribution can be computed by inverting a now-dimensional matrix [23, 24] (See Fig. 1). In one special case, inference will correspond to linearly interpolating between the representations of initial state and final state. Section 5 provides numerical experiments.
2 Related Work
Representations for time-series data. In applications ranging from robotics to vision to NLP, users often want to learn representations of observations from time series data such that the spatial arrangement of representations reflects the temporal arrangement of the observations [10, 16, 25, 26).
Followwig prior work [18, 19]. will use planning refer the problem of inferring intermediate states. not to refer to an optimal control problem.
'This probabilistic model is equivalent to discretized Omstten- Uhlenbeck process [21] and is also known an AR(1) model [22, Eq. 3.1.

---

Ideally, these representations should retain information required to predict future observations and infer likely paths between pairs of observations. Many approaches use an autoencoder, learning rep- resentations that retain the bits necessary to reconstruct the input observation, while also regularizing the representations to compressed or predictable [6, 27-31]. A prototypical method is the sequential VAE [5], which is computationally expensive to train because of the reconstruction loss, but is easy to use for inference. Our work shares the aims of prior prior methods that attempt to linearize the dynamics of nonlinear systems [32-37], including videos [38, 39]. Our work aims to retain uncertainty estimates over predictions (like the sequential VAE) without requiring reconstruction. Avoiding reconstruction is appealing practically because it decreases the computational requirements and number of hyperparameters, and theoretically because means that representations only need to retain bits about temporal relationships and not about the bits required to reconstruct the original observation.
Contrastive Learning. Contrastive learning methods circumvent reconstruction by learning repre- sentations that merely classify if two events were sampled from the same joint distribution [17, 40, 41] When applied to representing states along trajectories, contrastive representations learn classify whether two points lie on the same trajectory or not [10, 25, 42-44). Empirically, prior work in computer vision and NLP has observed that contrastive learning acquires representations where interpolation between representations corresponds to changing the images in semantically meaningful ways [16, 45-49].
Our analysis will be structurally similar to prior theoretical analysis on explaining why word embed- dings can solve analogies [50-52]. Our work will make Gaussianity assumption similar to Arora et al. [51] and our Markov assumption is similar to the random walks analyzed in Arora et al. [51], Hashimoto et al. [53]. Our paper builds upon and extends these results to answer questions such as: "what is the distribution over future observations representations?" and "what is the distribution over state (representations) that would occur the path between one observation and another?" While prior work is primarily aimed at explaining the good performance of contrastive word embed- dings (see, e.g., [)),), we are primarily interested in showing how similar contrastive methods are an effective tool for inference over high-dimensional time series data. Our analysis will show how representations learned via temporal contrastive learning (i.e., without reconstruction) are sufficient statistics for inferring future outcomes and can be used for performing inference on graphical model (a problem typically associated with generative methods).
Goal-oriented decision making, Much work on time series representations is done in service of learning goal-reaching behavior. an old problem [54, 55] that has received renewed attention in recent years [56-63]. Some of the excitement in goal-conditioned RL is reflection of the recent success of self-supervised methods in computer vision [64] and NI.P [65]. Our analysis will study variant of contrastive representation learning proposed in prior work for goal-conditioned RI. [42, 43]. These methods are widespread, appearing as learning objectives for learning value functions [26, 43, 66-73], as auxiliary objectives [71, 74-79]. in objectives for model-based RL [32, 80-82], and in exploration methods [83, 84]. Our analysis will highlight connections between these prior methods, the classic successor representation [85, 86), and probabilistic inference.
Planning. Planning lies at the core of many RL and control methods, allowing methods to infer the sequence of states and actions that would occur if the agent navigated from one state to goal state. While common methods such as PRM [87] and RRT [88] focus on building random graphs, there is a strong community focusing planning methods based probabilistic inference [19, 89, 90]. The key challenge is scaling to high-dimensional settings. While semi-parametric methods make progress on this problem this limitation through eani-parametric planning [91-93], it remains unclear how to scale any of these methods to high-dimensional settings when states do not lie on a low-dimensional manifold. Our analysis will show how contrastive representations may lift this limitation, with experiments validating this theory on 39-dimensional and 46-dimensional tasks.
3 Preliminaries
Our aim is to learn representations of time series data such that the spatial arrangement of rep- resentations corresponds to the temporal arrangement of the underlying data: if one example oc- curs shortly after another, then they should be mapped to similar representations. This problem setting arises in many areas, including video understanding and reinforcement learning. To de-
3

---

fine this problem formally, we will define Markov process with states indexed by time t: p(Z1.T zo) II p(x++1 Iz). The dynamics p(zz+1 | æz) tell us the immediate next state, and we can define the distribution over states t steps in the future by marginalizing over the intermediate states, pr zu) p(z1:( zn) d A key quantity of interest will be the d-disouunted state occupancy measure, which corresponds to a time-averaged distribution over future states:
(1+ = z) -1 (1 7)
p'el x).
(1)
t=
Contrastive learning. Our analysis will focus applying contrastive learning particular data distribution. Contrastive learning [10, 40, 94] acquires representations using "positive" pairs (z, xt) and "negative" pairs (Ir, ). While contrastive learning typically learns just one representation, will use two different representation for the two elements of the pair; that is, our analysis will use terms like ((x), (b(xrt) and (b(x)) We assume all representations lie in T
The aim of contrastive learning is to learn representations such that positive pairs have similar representations (ő(x) & (x(x))) while negative pairs have dissimilar representations (Ø(z) o (h(x)) Let p(æ, xx) be the joint distribution over positive pairs (i.c., (I, (t)x+)= ~ p(x,,+)) )). We will use the product of the marginal distributions to sample negative pairs ((x, p(z)p(=)). Let B
be the batch size, and note that the positive samples z. at index j in the batch serve as negatives for n; for any i # j. Our analysis is based on the infoNCE objective without resubstitution [10, 11]:
B
max E (( (().v())
Σ
log c ( oct/11 + log
(2)
Ejsre -blole---
Eieee - -1100133--0 1
We will use the symmetrized version of this objective [17], where the denominator is the sum across rows of a logits matrix and once where it is sum across the.
While contrastive learning is typically applied to an example x and an augmentation x+ ~ p(æ x) of that same example (e.g., a random crop). we will follow prior work [10. 42] in using the time series dynamics to generate the positive pairs, so xt will be an observation that occurs temporally
after æ. While our experiments will sample positive examples from the discounted state occupancy
measure (r+ ~ Pt++ (Ia+ z)) in line with prior work [43], our analysis will also apply to different distributions (e.g., always sampling a state k steps ahead).
While prior work typically constrains the representations to have a constant norm (i.e., to lie on the unit hypersphere) [10], we will instead constrain the expected norm of the representations is bounded, a difference that will be important for our analysis:
I Epp(s)) [1)) S.
Because the norm scales with the dimension of the representation, we have scaled down the left side by the representation dimension, k. In practice, we will impose this constraint by adding a regularization term X E,(a)) [=)]( to the infoNCE objective (Eq. 2) and dynamically tuning the weight X via dual gradient descent.
3.1
Key assumptions
This section outlines the two key assumptions behind our analysis, both of which have some theoretical justification. Our main assumption examines the distribution over representations:
Assumption 1. Regularized, temporal contrastive learning acquires representations whose marginal distribution representations p()) & p(z) (V(()) = v6) dx is an isotropic Gaussian distribution:
p( N(b 0, a= c-
In Appendix A. we extend prior work [20] provide some theoretical intuition for why this assumption should hold: namely, that the isotropic Gaussian is the distribution that maximizes entropy subject to an expected L2 norm constraint (Eq. 3) [95-97]. Our analysis also assumes that the learned representations converge to the theoretical minimizer of the infoNCE objective:
"This can be extended to controlled Markov processes appending the previous action to the observations.

---

Assumption 2. Applying contrastive learning to the symmetrized infoNCE objective results in representations that encode probability ratio:
= Pe+ (x++ zo)
(5)
p(x)C
This assumption holds under ideal conditions [98, 99] (see Appendix A.5), but we nonetheless call this an "assumption" because it may not hold in practice due to sampling and function approximation error. This assumption means the learned representations are sufficient statisties for predicting the probability (ratio) of future states: these representations must retain all the information pertinent to reasoning about temporal relationships, but need not retain information about the precise contents of the observations. As such, they may be much more compressed than representations learned via reconstruction.
Combined, these assumptions will allow us to express the distribution over sequences of represen- tations Gauss-Markoy chain. The denominator in Assumption 2, p(x), may have complex distribution, but Assumption 1 tells us that the distribution over representations has simpler form. This will allow us to rearrange Assumption to express the conditional distribution over representa- tions as the product of two Gaussian likelihoods. Note that the left hand side of Assumption 2 already looks like Gaussian likelihood.
Contrastive Representations Make Inference Easy
In this section, our main result will be to show how representations learned by (regularized) contrastive learning are distributed according to Gauss-Markov chain, making it straightforward to perform inference (c.g., planning, prediction) over these representations. Our proof technique will combine (known) results about Gaussian distributions with (known) results about contrastive learning. We start by discussing an important choice of parametrization (Section 4. 1) that facilitates prediction (Section 4.2) before presenting the main result in Section 4.3.
4.1 A Parametrization for Shared Encoders
This section describes the two encoders (+h()), o()) to compute representations of and z+_ While prior work in computer vision and NLP literature use the same encoder for both and x+, this decision does not make sense for many time-series data as it would imply that our prediction for p(x; wo) is the same as our prediction for p(z0 | 2.)). However, the difficulty of transiting from to (e.g., climbing to the peak of mountain) might be more difficult than the reverse (e.g., sledding down mountain). Our proposed parametrization will handle this asymmetry.
((x1) = (r)(wa)
16 (xre)
(((1))
Figure 2: A parametrization for temporal contrastive learn- ing.
We will treat the encoder v-()) as encoding the contents of the state. We will additionally learn matrix so that the function & + Au corresponds to (multi-step) prediction of the future representation.
To map this onto contrastive learning, we will use Φ(x) Av(xz) as the encoder for the initial state. One way interpreting this encoder as an additional linear projection applied on top of ( design similar to those used in other areas of contrastive learning [41]. Once learned, we can use these encoders to answer questions about prediction (Section 4.2) and planning (Section 4.3).
4.2 Representations Encode Predictive Model
Given an initial state zo. what states are likely to occur in the future? Answering this question directly in terms of high-dimensional states is challenging, but our learned representations provide a straightforward answer. Let 4/o = <((o) and 4/2+ v be random variables representing the representations of the initial state and future state. Our aim is to estimate the distribution over these future representations, p(te++ th)). We will show that the learned representations encode this distribution.
'While the result of Ma and Collins [98] has C(x) depending the symmetrized version [17] removes the dependence
5

---

Lemma 1. Under the assumptions from Section 3, the distribution over representations future states follows Gaussian distribution with mean parameter given by the initial state representation:
pla++ = vbb) = =N
Awo. E 1
The main takeaway here is that the distribution over future represen- tations has convenient, closed form solution. The representation norm constraint, c, determines the shrinkage factor o+1 E [0, 1); highly regularized settings (small c) move the mean closer towards the origin and decrease the variance, as visualized in Fig. 3. Re- gardless of the constraint c, the predicted mean a linear function + _ Av.. The proof is in Appendix A.2. The proof technique is similar to that of the law of the unconscious statistician.
c
Ash(1r)
Ash(ar)
((r)
4.3 Planning over One Intermediate State
We now show how these representations can be used for specific type of planning: given an initial state ze and future state Zi+, infer the representation of an intermediate "waypoint" state The next section will extend this analysis to inferring the entire sequence of intermediate states. We assume To - - Z++ form a Markov chain where Tar ~ p(2++ xo To) and Tt+ ~ p(x++ Zo Iww) are both drawn from the discounted state occupancy measure (Eq. 1). Let random variable the ((xw) be the representation of this intermediate state. Our main result that the posterior distribution over waypoint representations has closed form solution in terms of the initial state representation and future state representation:
Figure 3: Predicting represen-
tations of future states.
Theorem 2.
Under Assumptions I and 2, the posterior distribution over waypoint representations is Gaussian whose mean and covariance are linear linearfunction functions of the initial and final state representations:
p(u vo, v++) = =N (wwiiwwwi - E(A tx+ + Arto) 2
*++++1 1
The proof (Appendix A.3) uses the Markov property together with Lemma 1. The main takeaway from this lemma is that the posterior distribution takes the form simple probability distribution (a Gaussian) with parameters that are linear functions of the initial and final representations.
We give three examples to build intuition:
Example 1: A I and the is very large (little regularization). Then, the covariance is Σ-1 27 and the mean is the simple average of the initial and final representations μ (wo + v4+)). ). In other words, the waypoint representation the midpoint of the line v/o + 1h++-
Example 2: A is a rotation matrix and e very large. Rotation matrices satisfy AT - so the covariance is again -'1 ss 21. As noted in Section 4.2, we can interpret Axdo prediction of which representations will occur after v'o. Similarly, A A- = A " is a prediction of which representations will occur before v4+. Theorem 2 tells that the mean of the waypoint distribution the simple average these two predictions, μ == th Atto).
Example 3: A is rotation matrix and 0.01 (very strong regularization). In this case Σ-1 0.01 A ATA+ A+ 0.01+1 I ze 100/, so μ xs 100 (tho + ) = 0. Thus, in the case of strong regularization, 0.01+1 the posterior concentrates around the origin.
1
0.01
4.4 Planning over Many Intermediate States
This section extends the analysis to multiple intermediate states. Again, we will infer the posterior distribution of the representations of these intermediate states, tin Bargy ... We assume that these states form a Markov chain.
Theorem 3. Given observations from Markov chain To - 21 Itt, the joint distribution
representations a Gaussian distribution. Using V/n. = (Ways' vm.) to denote the concatenated representations each observation, we can write this distribution as
- p( (())))) x exp( 2 1 Σ- 3/nn + +n7 (Tvn)))

---

where E is tridiagonal matrix
Auoo 0
c+1 A 1
- A" +t - A
E-1=
A
A
and 1
A v2+
This distribution can be written the canonical parametrization X A- and μ = Ey. Recall that Gaussian distributions are closed under marginalization. Thus, once in this canonical parametrization, the marginal distributions can be obtained by reading off individual entries of these parameters:
p(u to, thx+)
gh:s u4 (20)) , (A (A-1)(.))
The key takeaway here is that this posterior distribution over waypoints is Gaussian, and it has closed form expression terms the initial and final representations (as well as regularization parameter and the learned matrix A).
In the general case of intermediate states, the posterior distribution is
1+ Ei ll [| A& -
p( Den Vu, v2+)
where vimo = tho and Voo. 1/+. This corresponds to chain graphical model with edge potentials f(0,, v²) =e
of Ap AV-V'13
Special case. To build intuition, consider the special case where A is a rotation matrix and is very large, so tt A"A"A+ A + et1 x 21. In this case, --1 is a (block) second difference matrix [100]:
21 -1
The inverse this matrix has a closed form solution [101, Pg. 471], allowing us to obtain the mean of each waypoint in closed form:
14 = (1 (1)) A() &
where A(i) .+ Thus, each posterior mean is convex combination of the (forward prediction from the) initial representation and the (backwards prediction from the) final representation. When A the identity matrix, the posterior mean is simple linear interpolation between the initial and final representations!
5 Numerical Simulation
We include several didactic experiments to illustrate our results. All results and figures can be repro- duced by running make in the source code: https ///gtthub. com/ com/vivee htt//vvvveeerrrreeeeeeeeeeeeeeee cttrs//ootrsttio planning. The expected compute time is a few hours on A6000 GPU, Figures in this section show error across different training and dataset split seeds.
5.1 Synthetic Dataset
To validate our analysis, we design time series task with 2D points where inference over intermediate points (i.e., in- -illlinggg requires nonlinear interpolation. Fig. 4 (Top Left) shows the dataset of time series data, starting the origin and spiraling outwards, with each trajectory using a randomly-chosen initial angle. We applied contrastive learning with the parametrization in Section 4.1 to these data and used the learned representations solve prediction and planning problems (see Fig. 4 for details). Note that these predictions correctly handle the nonlinear structure of these data states nearby the initial state in Euclidean space that are not temporally adjacent are assigned low likelihood.
5.2 Solving Mazes with Inferred Representations
Our next experiment studies whether the inferred representations are useful for solving a control task. We took 2d maze environment and dataset from prior work (Fig. 5, Left) [102] and learned encoders
7

---

DATASET
FORWARD PREDICTION
start (Xa)
(X7)
BACKWARD PREDICTION
PLANNING
Figure 4: Numerical simulation of our analysis. (Top Left) Toy dataset of time-series data consisting of many outwardly-spiralinn trajectories. We apply temporal contrastive learning to these data. (Top Right) For three initial observations ), we use the learned representations to predict the distribution over future observations. Note that these distributions correctly capture the spiral structure. (Bottom Left) For three observations (+), we use the learned representations to predict the distribution over preceding observations. (Bottom Right) Given initial and final observation, we plot the inferred posterior distribution over the waypoint (Section 4.3). The representations capture the shape of the distribution.
Maze Planning Success Rates
0.8
Success Rate
0.6
0.4
0.2
0
2
10
Initial & Distance Goal
contrastive (ours)
VIP
PCA
autoencoder
planning
Using inferred paths over our contrastive representations for control boosts success rates by 4.5x on the most difficult goals (18% - 84%). Alternative representation learning techniques fail to improve performance when used for planning.
Figure 5:
from this dataset. To solve the maze, we take the observation of the starting state and goal state, compute the representations of these states, and use the analysis Section 4.3 to infer the sequence of intermediate representations. We visualize the results using nearest neighbor retrieval (Fig. 5, Left). Figure 7 contains additional examples.
Finally, we studied whether these representations are useful for control. We implemented a simple proportional controller for this maze. As expected, this proportional controller can successfully navigate to close goals, but fails to reach distant goals (Fig. 5, Right). However, if we use the propor- tional controller track series waypoints planned using our representations (i.e., the orange dots shown in Fig. (Left)), the success rate increases by up to 4.5x. To test the importance of nonlinear representations, we compare with "PCA" baseline that predicts waypoints by interpolating between the principal components of the initial state and goal state. The better performance of our method indicates the importance of doing the interpolation using representations that are nonlinear functions of the input observations. While prior methods learn representations to encode temporal distances,

---

validation set that MSE of waypoint prediction for door opening
planning
AE planning
PCA planning
planning
contrastive planning (ours)
2
Waypoint MSE
contrastive planning (ours)
planning
VIP planaing
27
20
Figure 6: Planning for 39-dimensional robotic door opening. (Top Left) We dataset of trajectories demonstrating door opening from prior work [102] to learn representations. (Top Right) We method and three baselines infer intermediate waypoint between the first and last observation in a trajectory from held-out validation set. Errors are measured using the mean squared error with the true waypoint observation; predicted representations are converted observations using nearest neighbors on a validation set. (Boltom) We visualize a TSNE [103] of the states along the sampled trajectory blue circles, with the transparency indicating the index along the trajectory. The inferred plan is shown as red circles connected by arrows. Our method generates better plans than alternative representation learning methods (PCA. VIP).
unclear whether these methods support inference via interpolation. To test this hypothesis, one of these methods ("VIP" [69]) as a baseline. While the VIP representations likely encode similar bits as representations, the better performance of the contrastive representations indicates that the VIP representations do not expose those bits a way that makes planning easy.
5.3 Higher dimensional tasks
In this section provide preliminary experiments showing the planning approach in Section 4 scales to higher dimensional tasks. We used two datasets from prior work [102]: door hoor-human-tte -99-iimensional observations) and -hmmer-hummn encoders on these tasks, we evaluated the inference capabilities of the learned representations. Given the first and last observation from trajectory in validation set, use linear interpolation (see Eq. 7) to infer the representation five intermediate waypoint representations.
466diimensional observations). After learning
We evaluate performance in two ways. Quantitatively, we measure the mean squared error between each of the true waypoint observations and those inferred by our method. Since our method infers representations, rather than observations, we can measure errors in the space observations. Qualitatively, we visualize the high-ddimensioal observations from the validation trajectory using 2-dimensional TSNE [103] embedding, overlying the infer waypoints from our method: as before, we convert the representations inferred by our method to observations using nearest neighbors.
use nearest-neighhor retrieval
We compare with three alternative methods Fig. 6. To test the importance of representation learning. we first naïvely interpolate between the initial and final observations ("no planning"). The poor performance of this baseline indicates that the input time series are highly nonlinear. Similarly, interpolating the principle components of the initial and final observations ("PCA") performs poorly, again highlighting that the input time series highly nonlinear and that our representations are doing more than denoising (i.c., discarding directions of small variation). The third baseline. "VIP" [69],

---

learns representations to encode temporal distances using approximate dynamic programming. Like our method, VIP avoids reconstruction and learns nonlinear representations of the observations. However, the results in Fig. 6 highlight that VIP's representations do not allow users to plan by interpolation. The error bars shown in Fig. 6 (Top Right) show the standard deviation over 500 trajectories sampled from the validation set. For reproducibility, we repeated this entire experiment on another task, the 46-dimensional commer-human vo from D4RL. The results, shown in Appendix Fig. 8, support the conclusions above. Taken together, these results show that procedure for interpolating contrastive representations continues to be effective on tasks where observations have dozens of dimensions.
6 Discussion
Representation learning is at the core of many high-dimensional time-series modeling questions, yet how those representations are learned is often disconnected with the inferential task. The main contribution this paper is to show how discriminative techniques can be used to acquire compact representations that make it easy to answer inferential questions about time. The precise objective and parametrization we studied not much different from that used in practice, suggesting that either our theoretical results might be adapted to the existing methods, or that practitioners might adopt these details so they can use the closed- foosedd-ormmmo solutions to inference questions. Our work may also have implications for studying the structure of learned representations. While prior work often studies the geometry of representations as a post-hoc check, our analysis provides tools for studying when interpolation properties guaranteed to emerge, well as how learn representations with certain desired geometric properties.
Limitations. Our analysis hinges on the two assumptions mentioned in Section 3.1, and remains open how errors in those approximations translate into errors in our analysis. One important open question is whether it is always possible to satisfy these assumptions using exxpciennll-exxeessvvve representations.
Acknowledgments
We thank Seohong Park, Gautam Reddy, Chongyi Zheng, and anonymous reviewers for feedback and discussions that shaped this project. This work was supported by Princeton Research Computing resources Princeton University. This work was partially supported by AFOSR FA9550-22-1-0023..
References
[1] Evangelos Theodorou, Jonas Buchli, and Stefan Schaal. A Generalized Path Integral Control
Approach Reinforcement Learning. Journal of Machine Learning Research, 11:3137-3181, 2010.
[2] Hannes Jónsson, Greg Mills, and Karsten W Jacobsen. Nudged Elastic Band Method for Finding Minimum Energy Paths of Transitions. In Classical and Quantum Dynamics in Condensed Phase Simulations, pp. 385-404. World Scientific, 1998.
[3] Wouter Saelens, Robrecht Cannoodt, Helena Todorov, and van Saeys. A Comparison of Single-Cell Trajectory Inference Methods. Nature Biotechnology. 37(5):547-55 2019.
[4] Steven R Majewski, Ricardo P Schiavon, Peter M Frinchaboy, Carlos Allende Prieto, Robert Barkhouser, Dmitry Bizyaev, Basil Blank, Sophia Brunner, Adam Burton, Ricardo Carrera, al. The Apache Point Observatory Galactic Evolution Experiment (APOGEE). Astronomical Journal, 154(3):94, 2017.
[5] Shengjia Zhao, Jiaming Song, and Stefano Ermon. Towards Deeper Understanding of Varia- tional Autoencoding Models. arXiv:1702.00655. 2017.
[6] Yizhe Zhu, Martin Rengiang Min, Asim Kadav, and Hans Peter Graf. S3vae: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 6538-6547, 2020.
[7] Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015.
Adversarial Autoencoders.
10

---

[8] Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Alex Lamb, Martin Arjovsky, Olivier Mas- tropietro, and Aaron Courville. Adversarially Learned Inference. In International Conference on Learning Representations. 2016.
[9] Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating Diverse High-Fidelity Images With Vq-VAE-2. Neural Information Processing Systems, 32, 2019.
[10] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning With Contrastive Predictive Coding. arXiv: I 807 00770, 2018.
[11] Kihyuk Sohn. Improved Deep Metric Learning With Multi-Class N-Pair Loss Objective.
Neural Information Processing Systems, 29, 2016.
[12] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved Baselines With Momentum Contrastive Learning. arXiv:2003.044: .04997, 2020.
[13] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive Multiview Coding. In Computer Vision-ECCV 2020: 16th European Conference, pp. 776-794. 2020.
[14] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised Feature Learning via Non-Parametric Instance Discrimination. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3733-3742. 2018.
[15] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Contrastive Learning of Visual Representations. In International Conference on Machine Learning. pp. 1597-1607. 2020.
[16] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient Estimation of Word Representations in Vector Space.
2013.
[17] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models From Natural Language Supervision. In International Conference on Machine Learning. pp. 8748 8763. 2021.
[18] Matthew Botvinick and Marc Toussaint. Planning as Inference. Trends in Cognitive Sciences, 16(10):485 488, 2012.
[19] Hagai Attias. Planning by Probabilistic Inference. In International Workshop on Artificial Intelligence and Statistics, pp. 9-16. 2003.
[20] Tongzhou Wang and Phillip Isola. Understanding Contrastive Representation Learning Through Alignment and Uniformity the Hypersphere. In International Conference on Machine Learning, pp. 9929-9939. 2020.
[21] George E Uhlenbeck and Leonard S Ornstein. On the Theory of the Brownian Motion. Physical Review, 3665):823, 1930.
[22] George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time Series Analysis: Forecasting and Control. John Wiley & Sons, 2015.
[23] Dmitry M Malioutov, Jason K. Johnson, and Alan S Willsky. Walk-Sums and Belief Propagation in Gaussian Graphical Models. Journal of Machine Learning Research, 7:2031-2064 2006.
[24] Yair Weiss and William Freeman. Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology. Neural Information Processing Systems, 12, 1999.
[25] Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, and Yin Cui. Spatiotemporal Contrastive Video Representation Learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6964-6974. 2021.
[26] Benjamin Eysenbach. Ruslan Salakhutdinov, and Sergey Levine. C-Learning: Learning to Achieve Goals via Recursive Classification. In International Conference on Learning Representations. 2020.
[27] Siddharth Karamcheti, Suraj Nair, Annie Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, and Percy Liang. Language- Driven Representation Learning for Robotics. 12776 2023.
[28] Seongmin Park and Jihwa Lee. Finetuning Pretrained Transformers Into Variational Autoen- coders. arXiv:2108.00 10..02468, 2021.
[29] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. BERT: Pre- Training of Deep Bidirectional Transformers for Language Understanding. In NAICL-HI,, pp. 4171-4186
11

---

2019.
[30] Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, et al. UniMASK: Unified Inference in Sequential Decision Problems. arXiv:221 1. 1.10669, 2022.
[31] Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. Recurrent Latent Variable Model for Sequential Data. In Neural Information Processing Systems, volume 28. 2015.
[32] Rui Shu, Tung Nguyen, Yinlam Chow, Tuan Pham, Khoat Than, Mohammad Ghavamzadeh, Stefano Ermon, and Hung Bui. Predictive Coding for Locally-Linear Control. In International Conference on Machine Learning, pp. 8862-8871. 2020.
[33] Manuel Watter, Jost Springenberg, Joschka Boedecker, and Martin Riedmiller. Embed to Control: A Locally Linear Latent Dynamics Model for Control From Raw Images. Neural Information Processing Systems, 28, 2015.
[34] Ershad Banijamali, Rui Shu, Hung Bui, Ali Ghodsi, et al. Robust Locally-Linear Controllable Embedding. In International Conference Artificial Intelligence and Statistics, pp. 1751-
1759. 2018.
[35] Brandon Cui, Yinlam Chow, and Mohammad Ghavamzadeh. Control-Aware Representa- tions for Model-Base Based Reinforcement Learning. In International Conference on Learning Representattons. 2020.
[36] Tung D Nguyen, Rui Shu, Tuan Pham, Hung Bui, and Stefano Ermon. Temporal Predictive Coding for Model-Based Planning in Latent Space. In International Conference on Machine Learning, pp- 8130-8139. 2021.
[37] Tung Nguyen, Rui Shu, Tuan Pham, Hung Bui, and Stefano Ermon. Non-Markovian Predictive Coding for Planning in Latent Space. 2020.
[38] Ross Goroshin, Michael F Mathieu, and Yann LeCun. Learning to Linearize Under Uncertainty. Neural Information Processing Systems, 28, 2015.
[39] Dinesh Jayaraman and Kristen Grauman. Slow and Steady Feature Analysis: Higher Or- der Temporal Coherence in Video. In JEEE Conference on Computer Vision and Pattern Recognition, pp. 3852-3861. 2016.
[40] Michael Gutmann and Aapo Hyvärinen.
-ontrasCntt Estimation: A New Estimation Principle for Unnormalized Statistical Models. In International Conference on Artificial Intelligence and Statistics, pp. 297-304. 2010.
[41] Xinlei Chen and Kaiming He. Exploring Simple Siamese Representation Learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750-15758, 2021.
[42] Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine, and Google Brain. Time-Contrastive Networks: Supervupe Learning From Video. In IEEE International Conference on Robotics and Automation, pp. 1134-1141, 2018.
[43] Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Russ Salakhutdinov. Contrastive Learning as Conl-Conditioned Reinforcement Learning. Neural Information Processing Systems, 35:35603-35620, 2022.
[44] Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, and Shuran Song. Xskill: Cross Embodiment Skill Discovery. In Conference on Robot Learning, pp- 3536-3555, 2023,
[45] Laurenz Wiskott and Terrence J. Sejnowski. Slow Feature Analysis: Unsupervised Learning of Invariances. Neural Computation, 14(4):715-770, 2002.
[46] Wei Yan, Ci-Siang Lin, Fu-En Yang, Yu-Jhe Li, and Yu-Chiang Frank Wang. Semantics- Guided Representation Learning With Applications to Visual Synthesis. International
Conference on Pattern Recognition, pp. 7181-7187. 2021.
[47] Alon Oring, Zohar Yakhini, and Yacov Hel-Or. Autoencoder Image Interpolation by Shaping the Latent Space. In International Conference on Machine Learning, pp. 8281-8290. 2021.
[48] Ying-Cong Chen, Xiaogang Xu, Zhuotao Tian, and Jiaya Jia. Homomorphic Latent Space Translation. In IEEE/CVF Conference on Computer
Interpolation for Unpaired Vision and Pattern Recognition, pp. 2403-2411. 2019.
12

---

[49] Xiaofeng Liu, Yang Zou, Lingsheng Kong, Zhihui Diao, Junliang Yan, Jun Wang, Site Li, Ping Jia, and Jane You. Data Augmentation via Latent Space Interpolation for Image Classification. In International Conference Puttern Recognition, pp. 728-733. 2018.
[50] Omer Levy and Yoav Goldberg. Linguistic Regularities in Sparse and Explicit Word Repre- sentations. In Compulational Natural Language Leurning, pp. 171-180. 2014.
[51] Sanjeev Arora, Yuanzhi Li, Yingyu Liang. Tengyu Ma, and Andrej Risteski. A Latent Variable Model Approach to Pmi-Based Word Embeddings. Transactions of the Association for Computational Linguistics, 4:385-399, 2016.
[52] Carl Allen and Timothy Hospedales. Analogies Explained: Towards Understanding Word Embeddings. International Conference on Machine Learning, pp. 223-231, 2019.
[53] Tatsunori B Hashimoto, David Alvarez-Melis, and Tommi S Jaakkola. Word Embeddings as Metric Recovery in Semantic Spaces. Transactions of the Association for Computational Linguistics, 4:273-286, 2016.
[54] Allen Newell, John C Shaw, and Herbert A Simon. Report on a General Problem Solving Program. In IFIP Congress, volume 256, p. 64. 1959.
[55] John E Laird, Allen Newell, and Paul S Rosenbloom. Soar: An Architecture for General Intelligence. Artificial Intelligence, 33(1):1-64, 1987.
[56] Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision Transformer: Reinforcement Learning via Sequence Modeling. Neural Information Processing 3 34:15084-15097, 2021.
[57] Elliot Chane-Sane, Cordelia Schmid, and Ivan Laptev. Goal-Conditions Reinforcement Learning With Imagined Subgoals. In International Conference on Machine Learning, pp-
1430-1440. 2021.
[58] Cédric Colas, Tristan Karch, Olivier Sigaud, and Pierre- Oudeyer. Intrinsically Motivated -ond-Condi Reinforcement Learning: A Short Survey. Preprint, 2021.
[59] Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun, Meng Fang, Yali Du, Xiu Li, Lei Han, and Chongjie Zhang. Rethinking Goal-Condin Conditonnd Supervised Learning and Its Connection to Offline RL. In International Conference on Learning Representations. 2021.
[60] Yecheng Jason Ma, Jason Yan, Dinesh Jayaraman, and Osbert Bastani. How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression. arXiv:2206.0302. 2022.
[61] Yannick Schroecker and Charles Ishell. Universal Value Density Estimation for Imitation Leariing and Goal-Conditioned Reinforcement Learning. arXiv:2002.06473. 2020.
[62] Michael Janner, Qiyang Li, and Sergey Levine. Offline Reinforcement Learning as One Big Sequence Modeling Problem. Neural Information Processing Systems, 14:1273-1286, 2021.
[63] Joey Hejna, Jensen Gao, and Dorsa Sadigh. Distance Weighted Supervised Learning for Offline Interaction Data. arXiv:2304. 13774, 2023.
[64] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-Resolution Image Synthesis With Latent Diffusion Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. (0684-10695.. 2022.
[65] OpenAL. GPT-4 Technical Report. 12Xiv:2303.08772 2023.
[66] Chongyi Zheng, Benjamin Eysenbach, Homer Rich Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, and Sergey Levine. Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching From Offline Data. In International Conference on Learning Representations. 2024.
[67] Stephen Tian, Suraj Nair, Frederik Ebert, Sudeep Dasari, Benjamin Eysenbach, Chelsea Finn, and Sergey Levine. Model-Based Visual Planning With Self-Supervised Functional Distances. In International Conference on Learning Representations. 2020.
[68] Alekh Agarwal, Nan Jiang, Sham M Kakade, and Wen Sun. Reinforcement Learning: Theory and Algorithms. CS Dept. pp. 10-4, 2019.
[69] Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumnr, and Amy Zhang. VIP: Towards Universal Visual Reward and Representation via Value Implictt Pre-Training. In International Conference on Learning Representations. 2022.
13

---

[70] Yecheng Jason Ma, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. LIV: Saaguagg-Imaag Representations and Rewards for Robotic Control. International Conference Machine Learning, 2023.
[71] Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3M: Universal Visual Representation for Robot Manipulation. In Conference on Robot Leurning, pp. 892-909. 2023.
[72] Bo Liu, Yihao Feng, Qiang Liu, and Peter Stone. Metric Residual Network for Sample Efficient Goal-Conditioned Reinforcement Learning. AAAI Conference on Artificial Intelligence. volume 37, pp. 8799-8806. 2023.
[73] Tongzhou Wang, Antonio Torralba, Phillip Isola, and Amy Zhang. Optimal Reaching Reinforcement Learning via Quasimetric Learning. In International Conference Machine Learning. 2023.
[74] Max Schwarzer, Ankesh Anand, Rishab Goel, R Devon Hjelm, Aaron Courville, and Philip Bachman. Data-Efficient Reinforcement Learning With Self-Predictive Representations. In International Conference on Learning Representations. 2020.
[75] Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Avila Pires, Yash Chandak, Rémi Munos, Mark Rowland, Mohammad Gheshlaghi Azar, Charline Le Lan, Clare Lyle, et al. Understanding Self-Predictive Learning for Reinforcement Learning. In International Conference on Machine Learning, pp. 136322-36555 2023.
[76] Adam Stooke, Kimin Lee, Pieter Abbeel, and Michael Laskin. Decoupling Representation Learning From Reinforcement Learning. In International Conference Machine Learning, pp. 9870-9879. 2021.
[77] Homanga Bharadhwaj. Mohammad Babaeizadeh, Dumitru Erhan, and Sergey Levine. Infor- mation Prioritization Through Empowerment in Visual Model-Basee Basee RL. In International Conference on Learning Representations. 2021.
[78] Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre Côté, and R Devon Hjelm. Unsupervised State Representation Learning in Atari. Neural Information Processing Systems, 32, 2019.
[79] Pablo Samuel Castro, Tyler Kastner, P. Panangaden, and Mark Rowland. MICo: Improved Representations via Sampling-Basee Based State Similarity for Markov Decision Processes. In Neural Information Processing Systems. 2021.
[80] Raj Ghugare, Homanga Bharadhwaj. Benjamin Eysenbach, Sergey Levine, and Russ Salakhut- dinov. Simplifying Model-Based RL: Learning Representations, Latent-Space Models. and Policies With One Objective. In International Conference on Learning Representations. 2022.
[81] Cameron S. Allen. Learning Markov State Abstractions for Deep Reinforcement Learning. In Neural Information Processing Systems. 2021.
[82] Bogdan Mazoure, Benjamin Eysenbach, Ofir Nachum, and Jonathan Tompson. Contrastive Value Learning: Implicit Models for Simple Offline RL. In Conference on Robot Learning, pp. 1257-1267. 2023.
[83] Zhaohan Guo, Shantanu Thakoor, Miruna Pislar, Bernardo Avila Pires, Florent Altché, Corentin Tallec, Alaa Saade, Daniele Calandriello, -aantnntteen Grill, Yunhao Tang, et al. Byol-Explore Exploration by Bootstrapped Prediction. Neural Information Processing Systems, 35:3 1855- 31870, 2022.
[84] Yilun Du, Chuang Gan, and Phillip Isola. Curious Representation Learning for Embodied Intelligence. IEEE/CVE International Conference on Computer Vision, pp. 10388-10397, 2021.
[85] Peter Dayan. Improving Generalization for Temporal Differenne Learning: The Successor Representation. Neural Computation, 5:613-624, 1993,
[86] André Barreto, Will Dabney, Rémi Munos, Jonathan J Hunt, Tom Schaul, Hado P van Hasselt, and David Silver. Successor Features for Transfer in Reinforcement Learning. Neural Information Processing Systems, 30, 2017.
[87] Lydia E Kavraki, Petr Svestka. J-C Latombe, and Mark H Overmars. Probabilistic Roadmaps for Path Planning in High- Digeensimnsiin Configuration Spaces. IEEE Transactions on Robotics and Automation, 12(4))566- 58000 1996.
14

---

[88] Steven M LaValle, James J Kuffner, BR Donald, et al. Rapidly-Exploring Random Trees: Progress and Prospects. Algorithmic and Computational Robotics: New Directions, 5:293-308. 2001.
[89] Sep Thijssen and H. J. Kappen. Path Integral Control and Satee-Depnddett Feedback. Physical Review E, 91(3:0332104 2015.
[90] Grady Williams, Andrew Aldrich, and Evangelos Theodorou. Model Predictive Path Integral Control Using Covariance Variable Importance Sampling. arXiv: 509.01 149, 2015.
[91] Kuan Fang, Patrick Yin, Ashvin Nair, Homer Rich Walke, Gengchen Yan, and Sergey Levine. Generalization With Lossy Affordances: Leveraging Broad Offline Data for Learning Visuo- motor Tasks, In Conference on Robot Learning, pp. 106-117. 2023.
[92] Ben Eysenbach, Russ R Salakhutdinov, and Sergey Levine. Search on the Replay Buffer: Bridging Planning and Reinforcement Learning. In Neural Information Processing Systems, volume 32. 2019.
[93] Tianjun Zhang, Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine, and Joseph Gonzalez. C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks. In International Conference on Learning Representattins. 2021.
[94] Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khan- deparkar. Theoretical Analysis of Contrastive Unsupervised Representation Learning. In International Conference Machine Learning, pp.
2019.
[95] Claude Elwood Shannon. A Mathematical Theory of Communication. Bell System Technical Journal, 27(3):379 423,4422,-9-2 1948.
[96] E. T. Jaynes. Information Theory and Statistical Mechanics. Physical Review, 106(4):620-630
1957.
[97]
Keith Conrad. Probability Distributions and Maximum Likelihood. 2010.
[98] Zhuang Ma and Michael Collins. Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency. In Empirical Methods Natural Language Processing. 2018.
[99] Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On Variational Bounds of Mutual Information. In International Conference Machine Learning, pp. 5171- 5180. 2019.
[100] Nick Higham. What Is the Second Difference Matrix? https::/ htt:://nhiiiigg wtw.gghha.......ee 2022.
[101] Morris Newman and John Todd. The Evaluation of Matrix Inversion Programs, Journal of the
Society for Industrial and Applied Mathematics, 6(4):466-476, 1958.
[102] Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL; Datasets for Deep Data-Driven Reinforcement Learning. arXiv:2004.07219, 2020.
[103] Laurens Van der Maaten and Geoffrey Hinton. Visualizing Data Using T-SNE, Journal of Machine Learning Research, 9(11), 2008.
15

---

A Proofs
This section contains the proofs omitted from the main text.
A.1
Marginal Distribution over Representations is Gaussian
Recall Assumption 1, which states that the marginal distribution over representations is Gaussian. Assumption Regularized, temporal contrastive learning acquires representations whose marginal distribution representations p()) _ p(x) ((v(x)) = 1b) dæ is an isotropic Gaussian distribution: p( N (1
= c
We will motivate this statement by connecting the optimal contrastive infoNCE objective to the maximum entropy marginal distribution over the representations.
The infoNCE objective (Eq, 2) can be decomposed into alignment term and uniformity term [20]. where the uniformity term can be simplified as follows:
Ex-p(*)
log E ~p(x)
e
1lAp(( +1=))
N
- -[
log
e
N
N
1=1
j=l...Ni+
N
log
+
2 log(2
N
N
(2T) j=
1=1
N(u=V(e))EE=1)
k log(2nr)
log PGMM (vx()))
- H[v(x)) + log(2x).
The derivation above extends that in Wang and Isola [20] by considering Gaussian distribution rather than von Mises Fisher distribution. We are implicitly making the assumption that the marginal distributions satisfy p(x) p(x-), ). This difference corresponds to our choice of using negative squared L2 distance the infoNCE loss rather than an inner product, a difference that will be important later in analysis. A second difference is that we do not use the resubstitution estimator (i.e., we exclude data point z; from our estimate γGMM when evaluating the likelihood of a;), which we found hurt performance empirically. The takeaway from this identity is that maximizing the uniformity term corresponds to maximizing (an estimate of) the entropy of the representations.
We next prove that the maximum entropy distribution with an expected L2 norm constraint is Gaussian distribution. Variants of this result are well known [95-97], but we include full proof here for transparency.
Lemma 4. The maximum entropy distribution satisfying the expected norm constraint Eq. (3) is multivariate Gaussian distribution with mean μ O and covariance Σ c-
Proof. We start by defining the corresponding Lagrangian, with the second constraint saying that p(x) must be a valid probability distribution.
((p) H4, [x] + A (Epta) [|| -c k) + 22 (f p(x) (x) dx -1) 11)
We next take the derivative w.r.t. px=):
ac
0p(x:) -pxx)/p(1) - log A1 A1|l0o13 Aa
Setting this derivative equal to 0 and solving for p(x), get
p(z) = 1+ 1111=11
16

---

We next solve for λ and X2 to satisfy the constraints in the Lagrangian. Note that ~ NiH = 0, c. I) has an expected norm =| c. so we must have = 2c. as the normalizing constant for Gaussian, finally giving us:
We determine À
p())
1
(2cm kk2
corresponding to an isotropic Gaussian distribution with mean μ and covariance Σ I.
A.2 Proof of Lemma
Below we present the proof of Lemma 1.
Lemma 1. Under the assumptions from Section 3, the distribution over representations of future states follows a Gaussian distribution with mean parameter given by the initial state representation:
vh) = NN
Adoo -= 11
plve+
Proof. Our proof technique will be similar to that of the law of the unconscious statistician:
p(dr+ vo) (m) p(urt, vo)
x ff p(ertt, zet, do xo) dz dæy
/f pluee (010 xo)p(xe+ zo)p(z1) dæt+ dæo
(c) // ((b(100+))) - vh+ (())))))==))) ((v((00) volp(ar+)e 211A0 (22 dae dro
c i1lAvo-vie //((((E))))) = thet) ((b((a))) = vo)p(xet )p(zo) dx dzo
(Dleewt (1b()x+))))))))))) dz1+ (
e A + 11
p(zo) (1b(zz) dæo
p(n))
p(Vo)
g e 1AAo vc- v1 1o ll Ivol3
2 lii I Ave Avo-veell
( e
N (14i ct ANo, E= cfs1)
In (a) we applied Bayes' Rule and removed the denominator, which is constant w.r.t. 3/+ In (b) we factored the joint distribution. noting that v'r and th are deterministic functions of r4+ and zo respectively, so they are conditionally independent from the other random variables. In (c) we used Assumption 2 after solving for p(a+- æo)= p(x++)e
- In (d) we noted that when the integrand is nonzero, takes on constant value of e à (lAAvo Salio-vesd w we can move that constant outside the integral. In (e) we used the definition of the marginal representation distribution (Eq. 6). In (f) we used Assumption to write the marginal distributions p(t++) and p( Pu) as Gaussian distributions. We removed the normalizing constants, which are independent of dit. In (g) we completed the square and then recognized the expression as the density of multivariate Gaussian distribution.
A.3 Proof of Theorem 2: Waypoint Distribution
Theorem Under Assumptions I and 2, the posterior distribution over waypoint representations is Gaussian whose mean and covariance are linear functions of the initial and final state representations:
plua voo vi++ = N (V E(AT th + E 1
17

---

Proof.
(a) p(1++ Men )p( 150)
p(/ I soo sl++)
plut Vo)
(b) clf Anw Me ll 2 lef, A80 vw
11
(c)
x e 200 (110-1))23) x (vo -p) N (1 2)
where E-
c4I A + I and μ = E(A + + Avho). A"
In line (a) we used the definition of the conditional distribution and then simplified the numerator using the Markov property. Line (b) uses the Lemma 1. Line (c) completes the square, the details of which are below:
c+1
_
c
AuDo - vull?)
Ar - ui+
+
c
c
+
2
c
c+1 1
T
c+1 A)²
c
)
A U/n - 20/7
(-ETA) A) Viw + 2h v'+.
2
c+1
+ 40
FFI A)
c
( A 400 {
T
2010
(-c_A)
A)
Vw + wi twe
c
c
cons..
1
AT + 1 We
" (AT a/e+ + Ardo) wm
2
c+1
c+
1
A c+1,
thin - (AT + Avho) "
2
7-1
const.
(1' p)T (v'e - 14),
where E-1 क TT للييع and μ = E(AT Mh+ + Avoo). Above, we have used equality up to an additive constant that is independent of y/u-
const.
denote
A.4 Proof of Theorem 3: Planning over Many Intermediate States
Theorem 3, Given observations from a Markov chain 2y - x1 ... Iet, the joint distribution over representations is a Gaussian distribution. Using Vien (Dew) .Dwn) to denote the concatenated representations of each observation, we can write this distribution as
p(Win)
E- + n1'))=n)
where E-1 a tridiagonal matrix
Auho 0
A
A +
1
- AT
c+1
=-1=
A
A
c+1
AT
and 1
AT wht
Proof. We start by recalling that the waypoints form a Markov chain, so we can express their joint density as product of conditional densities:
p(PI:)) = p(to))p(á1 vo)p((22 41)
p(pb (n-1))
The aim of this lemma is to express the joint distribution over multiple waypoints, given an initial and final state representation:
p(p1. Vo, w/++) (a) p(ahi (00)p(()))) v.)
p(u Vol
18

---

@2 p(p1 (oo)p((22 (b1) p(1r+ 1//)
(c)
x exp
1 cctl c+1
Agb - -
2 c c1 A
1 1ct1
2/2 12
1 n+1
Ath - 2/ l1
c
c+1
(d)
- 2.t 2 Awo + v0 A か1
1 c+ul vi 11
exp
2 A Arh + A
1otl 1/
c
2 A" A + 2 A0 v3
_ 0s
2ct1
c 1 AT Arh.n + t A
2 +ot Ve+
ca r
exp( 2 E
1m + 17 1/1))
where
c A c+1
A |
-A-
Auto
-A
AT A + 1
-AT A A
c+1
E-1=
-
c+1
and 17
c+1
AA7 +
In (a) we applied Bayes' rule and removed the denominator because it a constant with respect to vian- In (b) we applied the Markov assumption. (c) we used Lemma express the conditional probabilities as Gaussians, ignoring the proportionality constants (which are independent of q/. In (d)
we simplified the exponents, removing terms that do not depend on d'i...
A.5 Formalizing Assumption 2
Assumption relates the learned contrastive critic to log-likelioood ratio between the positive and negative data distribution.
Assumption Applying contrastive learning to the symmetrized infoNCE objective results in representations that encode probability ratio:
-(10(=))=+<=113 = Pe+ (I++ x xo)
p(x)C
We can justify this assumption by analyzing the general solution to the symmetrized version of the Oord al. [10] infoNCE objective, which we do in Lemma 5, Applying this lemma to our representation learning objective (2) for sufficiently large batch size then yields Eq. (5), with the function approximator [Ø(z)) (xx II? z f(z
Lemma 5. The solution to the optimization problem
B log
max lim E
Σ
Eis (ieinnt + log
clace!) Σjet (
(8)
f(a B
B
satisfies
xr)
f(x,x*) = log
p(x+)C
for some C.
Proof of Lemma 5. We first break down the LHS and RHS of Eq. (2):
B
el(nnzt)
of (
lim E [((e)))) 11 + max
log
+ log
f B-20
EjØz (()a))
Di ef (2si2t))
19

---

e
1
Ji(f) lim [((ex)
log
8-000
B
Σ jzt e(lesiet)
1
J2(f)
lim (u B-000
log
B
Ejzt j el(os127)
We now use the following result from Ma and Collins [98]:
Lemma 6. The optimal solutions fi and for Ti and J2 satisfy
xx log p(2; (x+) log (2))
(10)
f2(x, log p(x [x) log (2xx
(11)
for arbitrary (1 (2), c2(
For any C, when e1 (2) = Cp(z) and c2(xx)) ) Cp(x*t)
p((x) x²t) ((012²)) = log
p(x xz)
fi = log
fe(x,x*).
(12)
p(x)C'
pl
follows that Eq. (12) maximizes both Jn and J2, and is precisely the optimal solution Eq. (9) for Eq. (8).
What does C represent? From Eq. (9), we can connect to the mutual information I [xx, x
C = E(z, )aplatt) (f(,,) xx
(13)
1xxxx))
Proof of Lemma 6. We can first consider Jn without loss of generality. Denoting
g(x,x*)= el (x,wtt
we take the functional derivative:
B
g xxt) Σ Ejtt I g1ri 255)
8J (logg) = lim 8 B ((xxx))) B-:00
log
B
1=1
B
lim E [[ rpla,ot)
8(0,,0) X22 o(o. 1)
B
_g(2,
612 6m g( 11
lim E (1B
B-
B
g(
Σ Eser 2(10,00)
B
lim ((a, -plaa
Σ
)p(x (xs) (leittt
B-400
B
0(21111 0 )+ Eee al () )p(att) do
6glar
B
lim E
8g(2s, x
pla p g(z(.at)
B-400
B
i=1
E (1 Esniolane p(o+)) dx
20

---

Figure 7: Our approach enables goal-conditioned policy to reach farther targets (red) from the start
(green) by planning over intermediate waypoints (orange).
(a) Contrastive representations
(b) PCA representations
Figure 8: Planning for 46-dimensional robotic hammering. (Left) dataset of trajectories demonstrat- ing a hammer knocking nail into a board [102]. (Center) We visualize the learned representations blue circles, with the transparency indicating the index of that observation along the trajectory. We also visualize the inferred plan (Section 4.3) as red circles connected by arrows. (Right) Representa- tions learned by PCA the same trajectory (a, left).
B
lim B((a,at))) (1tt.et) mpla,ot) B-000
og at
p(a l(a) glasritt
p(x+)) dx
(ars De Es o(
P k((rz) indep, of
B
lim E ) B-
1
Σ
8g(2s, xt
pla (at)
k(x))p(x))) dx
B
glacnt))
1=1
f òg(z, xt ( k(x)p(x*) )) dx
This is zero when
g(x,x*)= = p(x at)
k(x)p(x))
i...
f(zxxt) = logp(x x+) log Ci (x)
k(x)p(>)
as in Eq. (10), and Eq. (11) follows similarly, exchanging or and t
B Additional Experiments
Fig. 7 visualizes the inferred waypoints from the task in Fig. 5. Fig. 8 visualizes the representations leamned 46-dimensional robotic hammering task (see Section 5.3).
21

---

B.1 Stock Prediction
We show results stock opening price task in Fig. 9.
AMPL
200
163
preetss
60
140
129
100
80
60
502
3000
500
10000
500
2000
1,000
ABBV
ABC
ABT
ACN
60
1.0
1.0 -
140
1/002 (.50)
80-
100
40
1030
ADBE
ADI
ADM
ADP
1.00
140
200
180
40
50
5
505
1000
505
1.000
5.00 1000
520
1.000
days
days
days
days
Figure 9: Stock Prediction. We apply temporal contrastive learning time series data of the stock market. Data are the opening prices for the 500 stocks in the S&P 500, over four year window. We remove 30 stocks that are missing data. For evaluation, validation set, and use Theorem 2 to perform "inpainting." predicting the intermediate stock prices jointly for all stocks (orange), given the first and stock price. The true stock prices are shown in blue. While we do not claim that this is state-of-the-an model for stock prediction, this experiment demonstrates another potential application of theoretical results.
choose a 100 day window from
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction are supposed by the Lemmas the main text.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See "Limitations" in Sec. 6, and discussion following the Assumptions in the main text.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess..
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer:
[Yes]
Justification: Proofs for each Lemma are provided in either the main text or the appendices.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: All results and figures in paper be reproduced by running make in the linked code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
24

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Code has been released. The datasets used have been released by prior work.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
cls.ss.ccppulicc///c httpppulllluudeeeeeeeppppppppeee guides/ CCodeSubmi ssi wwwsnnoolliii onlliccy)) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details are in the appendix and/or the supplemental code. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Sec. 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
25

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Sec. 5
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http::://errrpp..cpppplllt////// http://p.....p.ccccccccccooeeee.
Answer: [Yes]
Justification:
We are unaware of any violations.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper provides theoretical analysis of representations learned by con- trastive learning. This analysis suggests new ways of using these representations, for both beneficial and harmful purposes.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
26

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See citations to the D4RL dataset.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
27

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: N/A
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "CZSEMx3EMI", "reviewer": "Reviewer_4JuK", "summary": "This paper addresses the challenges of probabilistic inference with high-dimensional time series data. It provides compact, closed form solutions in terms of learned representations by extending a prior work to show that the learned representations can be modeled as a Gauss-Markov chain. It demonstrates the effectiveness of this theoretical result via numerical experiments on 39-dimensional and 46-dimensional tasks.", "questions": "The usefulness of the geometric properties remains somewhat abstract. While the special case provides some insights, it is not fully clear what it actually means for practical applications. Could you elaborate on how the geometric properties are used in practical scenarios and whether it has been used in your numerical experiments? Why linear interpolation can be used for your high-dimensional tasks while nonlinear interpolation is required for your synthetic task?\n\nI suggest separating Assumption 2 into the assumption inherited from the prior work and a statement of its results. Additionally, the term \"prior work\" is used throughout the paper, creating some ambiguity. It would enhance clarity by specifying which particular prior work is the one that shows \"the representations learned by contrastive learning encode a probability ratio\".\n\nAddressing the above two concerns can help understand the contributions better.", "limitations": "Yes, the authors have adequately addressed the limitations.", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "This paper clearly states the backgrounds and motivations. It builds upon prior work to derive a theoretical result that is useful for managing high-dimensional data in practical applications. It enhances reader understanding through examples that build intuition and illustrations that visualize methods. It also has comprehensive numerical experiments to not only validate theoretical results but also demonstrate the applicability to real-world applications.", "weaknesses": "The writing of this paper can be improved, particularly in the preliminary section, which is hard to follow due to a lack of self-contained explanations for some notations and definitions. For example, why use product of the marginal distributions to sample negative pairs and what does ≈ mean for similarity? What is the infoNCE objective without resubstitution, particularly what is f and B in (2)? Why the subscript of expectation in (2) seems to not match with the sampling rule (x, x+) ∼ p(x, x) mentioned earlier? What does TSNE in the experiment section stand for?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThanks for the detailed comments and suggestions for improving the paper. It is clear that a lot of time went into carefully reading the paper! It seems like the main suggestion was to clarify a few sections of the text, which we have done. Per the reviewer's suggestion, we have also clarified the implications of our work on practical applications. **Do these revisions, together with the answers below, fully address the reviewer's concerns?**\n\nKind regards,\n\nThe Authors\n\n\n> it is not fully clear what it actually means for practical applications. Could you elaborate on how the geometric properties are used in practical scenarios?\n\nOur theoretical analysis is broadly applicable to several practical applications. As noted by Reviewer 4BoG, our analysis suggests that a version of temporal contrastive learning can be used anywhere Kalman filters and similar probabilistic time series models are used today. Our analysis removes a key barrier in using these sorts of probabilistic models: whereas typically fitting these models to data requires learning a generative model (which need lots of parameters and compute), our analysis shows how _discriminative_ methods can be used to fit similar models (and the computer vision and language processing communities have already developed highly efficient algorithms and implementations of these discriminative methods). We will clarify this in the paper.\n\n> how are the geometric properties used in your numerical experiments?\n\nWe use the geometric properties of these representations in all of our experiments. In Fig. 4, we use these geometric properties to solve a retrieval problem: which states are likely to be visited in between two other states? In Fig. 5, we use these geometric properties to solve a planning problem: inferring a sequence of waypoints through a maze, and then using these waypoints to control an agent interacting with the maze.\n\n> Why linear interpolation can be used for your high-dimensional tasks while nonlinear interpolation is required for your synthetic task?\n\nWe use the math in Lemma 3 to perform the interpolation for all our tasks. This interpolation is linear with respect to the representations, but nonlinear with respect to the observations. We will modify the text to clarify. [Let us know if there was something else that was confusing here!]\n\n> Why use product of the marginal distributions to sample negative pairs?\n\nAlmost all contrastive learning papers sample negative examples in this way (Chen et al., 2020; He et al., 2020; Oord et al., 2019; Radford et al., 2021; Wu et al., 2018). Not only is this choice easy to implement (just sample pairs of states independently from dataset/replay buffer), but prior work also shows that it is highly effective.\n\n> what does ≈ mean for similarity?\n\nWe use ``$\\approx$'' when providing intuition about three worked examples in Sec. 4.3. Let's look at Example 1, where we assume $A = I$ and $c$ is large. Plugging these example values into Lemma 1, we get $\\Sigma^{-1} = \\frac{c}{c+1}I + \\frac{c+1}{c}$. As $c \\rightarrow \\infty$, we have $\\Sigma^{-1} \\rightarrow 2I$ (and $\\Sigma \\rightarrow \\frac{1}{2} I$). Thus, as $c \\rightarrow \\infty$, we have $\\mu \\rightarrow \\frac{1}{2} (\\psi_0 + \\psi_{t+})$.\n\nNB: There was a typo in the original version of Lemma 1: the mean should have been $\\mu = \\Sigma (A^T \\psi_{t+} + A \\psi_0)$ rather than $\\mu = \\Sigma^{-1} (A^T \\psi_{t+} + A \\psi_0)$. We have corrected this in our local copy of the paper, and use the correct version in the answer above.\n\n> What is the infoNCE objective without resubstitution\n\nThe infoNCE objective is a commonly used contrastive learning objective, often attributed to Oord et al. (2019) but which also appears in earlier work (Sohn, 2016). We use ``resubstitution'' to mean that the positive example $x^+$ doesn't appear in the denominator: while early work on contrastive learning typically included the positive term in the denominator, subsequent work found that excluding it can boost performance (Yeh et al., 2022). We have clarified these details in the paper\n\n> particularly what is f and B in (2)?\n\nB is the batch size; we have added this detail to the paper.\nWe have revised Eq. 2 to remove $f(\\cdot, \\cdot)$ and replaced it with $\\frac{1}{2}\\|\\|\\phi(x) - \\psi_s^+)\\|\\|_2^2$.\n\n> Why the subscript of expectation in (2) seems to not match with the sampling rule (x, x+) ∼ p(x, x) mentioned earlier?\n\nThanks for catching this typo, which we have fixed.\n\n> What does TSNE in the experiment section stand for?\n\nt-Distributed Stochastic Neighbor Embedding (). We have added a citation.\n\n> \"prior work\" is used throughout the paper, creating some ambiguity... Which particular prior work shows \"the representations learned by contrastive learning encode a probability ratio\"\n\nWe have addressed this concern by clarifying the particular prior works that these statements refer to. E.g., we provide a citation to (Ma & Collins, 2018; Poole et al., 2019) for the statement that \"the representations learned by contrastive learning encode a probability ratio\n\n**References**\n\nKaramcheti, et al. (2023). Language-Driven Representation Learning for Robotics. \n\nMa, et al. (2023). VIP: Towards universal visual reward and representation via value-implicit pre-training.\n\nMikolov, et al. (2013). Distributed Representations of Words and Phrases and their Compositionality.\n\nOliveira, P. (2006). Interpolation of signals with missing data using PCA.\n\nOord, et al. (2019). Representation Learning with Contrastive Predictive Coding.\n\nPark et al. (2021). Finetuning pretrained transformers into variational autoencoders. \n\nSermanet, et al. (2017). Time-Contrastive Networks: Self-Supervised Learning from Multi-view Observation\n\nvan der Maaten, et al. (2008). Visualizing Data Using t-SNE.\n\nYeh, et al. (2022). Decoupled contrastive learning.\n\nZhang, et al. (2023). Universal visual decomposer: Long-horizon manipulation made easy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u7bKpdWw12", "reviewer": "Reviewer_4BoG", "summary": "This paper proposes to use contrastive learning technique in time series feature learning, which are helpful for improving inference performance later on.", "questions": "Does such a contrastive learning method for time series require sampling from the environments repeatedly from any given starting state? If so, how is the applicability of this method on the environments that are only sampled once?", "limitations": "N/A", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method proposed here is simple but effective as shown in the numerical experiments. Employment of contrastive learning in time series control task has the potential as a complementary technique to the generative modeling methods that are currently popular.", "weaknesses": "Seeing the analysis of the learned feature unrolling as a Gauss-Markov process reminds me of the very classic Kalman filter. In fact, although Kalman filter is not designed to be learned in the discriminative way that this paper proposes, the equations in this paper share fair bit of similarity to those in Kalman filter. I would like to see some discussion on the difference that separates from this work from the prior time-series modeling with similar formulations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThanks for the detailed review, and for the insightful comment about the connection with Kalman filters. As noted by the reviewer, both Kalman filters and the representations that we analyze correspond to a probabilistic model of time series data. The key difference (again, noted by the reviewer) is that Kalman filters are fit using generative methods while we propose a method for fitting a similar generative model using discriminative methods. This finding is useful because discriminative methods avoid the need for a decoder, thus significantly reducing the number of parameters (the decoder typically has many more parameter than the encoder) and the compute required for training (the decoder typically requires more time for a forward pass than the encoder). We will add this discussion to the paper. **Together with the answers below, does this address any concerns the reviewer might have about the paper?**\n\nKind regards,\n\nThe authors.\n\n> Does such a contrastive learning method for time series require sampling from the environments repeatedly from any given starting state?\n\nNo – this contrastive learning method does not need to be initialized in a single fixed state, nor does it need to be initialized in every possible state. We expect that Assumption 2 will hold when the training data covers a range of different states; as with other machine learning applications, we expect that the learned representations will generalize if given sufficient data (so we do not expect that every state needs to be visited to ensure Assumption 2).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CZpWWSUSB0", "reviewer": "Reviewer_e5if", "summary": "This paper shows that learning contrastive representations with the infoNCE objective combined with an L2 constraint on these representations permit Gaussian distributions on the representations. The authors then show for timeseries data, that, when we make a specific Markov chain assumption (a kind of linear state-space model) then the distribution over learned representations permits a nice Gaussian form for all future or past representations. As a consequence of this, planning can be performed through simple linear interpolation within the learned space.\n\nThe paper shortly validates their method in a didactic experimental setup.", "questions": ".", "limitations": "1) the paper concludes that it provides a kind of 'recipe' for how to get models that have a linear interpolation property for planning, but aside from the model that the authors designed in a very specific way this is not discussed.\n\n2) the experiments are somewhat trivial since the authors expected baselines, that were not designed for a particular feature (planning through linear interp.) to suddenly have this feature.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is very well written, the presentation of theoretical results, their intuition, and how these are used is for a large part excellent.\n\nThe experiments nicely contribute to the main point of the method, that, within a contrastive learned space we can do planning in an effectively closed form rather than using an expensive dynamic programming method. The visualizations nicely show how the authors' method differ from the baselines.", "weaknesses": "### Summary\n\nI find the discussion and narrative of this paper too weak to confirm the novelty and contributions of the method. The authors make assumptions on their model such that it satisfies a nice practical property: planning through interpolation. However, the theoretical results that are presented should be well known consequences of Gaussian assumptions for linear state space models. The presentation makes this insight almost seem profound, but the weak discussion of related work makes me skeptical of the contribution. \n\nAlthough this paper presents a nice observation, taking all listed points of improvement into account, I don't think this paper is sufficient yet for an accept. But I'm open to improve the score if the authors address my concerns.\n\n### Major comments\n - Lemma 2 is stated as the main result. Although this way of defining waypoint conditionals is an interesting perspective on planning, the result should be a well-known consequence of using Gaussians (The same goes for Lemma 3). This makes it unclear what the novelty/ contribution of this result is. The authors need to improve this by discussing prior work/ results after introducing their model.\n - The title of the paper: inference via interpolation, is too hidden within the main text (at the end of 4.4 in Eq. 7). I would strongly suggest turning this result into a corollary (indeed with proof referring to Newman and Todd 1958 Pg. 471), then moving this result all the way to the top of section 4, and restructuring the text to move towards this result. \n - The interpolation in Eq. 7) only presents the mean-representation. There is no such discussion on the covariance.\n - The experiment of figure 5 is not well-designed. The introduced method makes use of strong assumptions that allow it to make use of linear-interpolation for planning towards future or past states. Still, the authors compare this to baselines where (also without discussion) we suddenly assume that these have the same functionality, why is this a reasonable assumption? As a result, a baseline without planning outperforms the baselines **with** planning.\n - Discussion Line 347 (last sentence) strongly oversells the presentation of the paper. The interpolation property is a nice consequence of the method's assumptions (Eq 3. and As.1, 2) and the contrastive learning. However, there is no discussion on how and why other methods cannot achieve similar things. My suggestion is to drop the 'when' comment, and only keep the 'how', because that's what the paper ultimately shows.\n\n### Technical comments\n - Eq 2. is confusing since it is first written that $(x, x^+) \\sim p(x, x)$ but in the Eq. it appears as $p(x)$. The term $p(x, x)$ is wrong, write this as $p(x, x^+)$ and keep its use consistent. Perhaps use $(x^+, x) \\sim p(x^+ | x)p(x)$ and $x^- \\sim p(x)$?\n  - Assumption 2, phrasing this result as an assumption weakens the story of the authors, especially since this is the equation that the authors' rely on for later analysis. It's usual to make assumptions or simplifications of the real world, so the fact that approximation error or model misspecification might not satisfy the result should not take away from it. I suggest making this a proposition and writing an assumption around this that discusses approximation error, please directly refer to the proof afterwards (citation + where to find it).\n  - Lemma 3, $\\eta$ isn't clearly shown to have zero entries apart from the first and last index. Why not use the same definition as in Appendix A.4?\n  - Missing proof of Lemma 3, no reference to appendix.\n\n### Minor comments\n - Line 166 typo: 'that that'\n- The two representations are $\\phi$ and $\\psi$ composed with $A$. Calling this two separate representations is confusing, there is only 1 learned representation and $A$ is used to learn the transport from embedding $\\psi(a)$ to $\\psi(b)$. Maybe state this upfront, i.e., in the first sentence line 187 say $\\psi(x)$ and $\\phi(x) = A\\psi(x)$, that helps when reading chronologically.\n - Line 215, refer back to Eq.3 for the definition of $c$\n - Line 231, proof of lemma 2 is in appendix A.3, not A.2.\n - The three examples in 4.3 for lemma 2 are nice to have for an unfamiliar audience, but I don't see how this contributes to the result since $A$ is learned, and will likely never satisfy trivial solutions.\n - Line 268, the interpolation result is nice, but the linear interpolation insight is not particularly profound (the text makes it seem as if it were so).\n - The start and end symbols in Figure 4 are too small to see and overlap with the black dots too much, even at 500% zoom.\n - Figures 5, 6 are not chronologically ordered/ numbered (6 appears before 5).\n - The choice of baselines and experiments are somewhat limited. I would have liked a comparison to a decoding method which the authors criticize in the introduction paragraph 2 (e.g., using the DreamerV2/ PlaNet RSSM model).\n - There is still space left on page 9, the authors could have spent more space on discussing the limitations of their method, which right now is rather shallow.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThanks for the detailed comments and suggestions for improving the paper. It's clear that a large amount of time was spent on this review – we appreciate it, and it will help improve the paper. We have added a new autoencoder baseline. To address the reviewer's concern about the novelty of the results, we propose a revision (see below) that would clearly delineate known results (joint Gaussianity) and new results (about temporal contrastive learning). Below, we also discuss several other revisions to the paper (e.g., clarifying the aim of the experiments), as well as answering questions. **Do these revisions and discussion fully address the reviewer's concerns about the paper?**\n\n> I would have liked a comparison to a decoding method\n\nAs suggested by the reviewer, we have added an additional autoencoder baseline to the  experiments from Figures 5 and 6, with the results shown in the rebuttal PDF (Fig. R2). As one might expect, this autoencoder baseline does not have the interpolation property.\n\n> Lemma 2 is a well-known consequence of using Gaussians\n\nThe key novelty in Lemma 2 is that we are analyzing the representations produced from temporal contrastive learning.  We agree that it is well known that, for a jointly Gaussian distribution, any marginal and posterior distribution is also Gaussian. The novel part of Lemma 2 is saying that the joint distribution over contrastive representations obeys a Gaussian distribution.\n\nWould it be more clear if we separated Lemma 2 into two results?\n* [not novel] the known result about jointly Gaussian distributions and their marginals/posteriors (citing prior work).\n* [novel] our new result about temporal contrastive representations being jointly gaussian\n\nIf the reviewer thinks it would clarify the writing, we can make a similar revision to Lemma 3.\n\n> The authors compare this to baselines we assume have the same functionality\n\nThanks for raising this concern. We believe that the experiment does give an important bit of information, but agree that the framing was confusing. We agree that it is unclear whether prior methods have the same functionality (though, there is some prior work that uses PCA for interpolation (Oliveira, 2006) and the VIP representations for generating subgoals (Zhang et al., 2023). At the same time, before working on this paper, we also believed that temporal contrastive methods didn't have this functionality. The aim of this experiment is to show that temporal contrastive methods do have this functionality, and that this functionality isn't caused by some artifact of the dataset. We will revise the paper to clarify.\n\n> My suggestion is to drop the 'when' comment, and only keep the 'how', because that's what the paper ultimately shows.\n\nThanks for the suggestion—we have made this revision.\n\n> The title of the paper: inference via interpolation, is too hidden within the main text (at the end of 4.4 in Eq. 7). I would strongly suggest turning this result into a corollary (indeed with proof referring to Newman and Todd 1958 Pg. 471), then moving this result all the way to the top of section 4, and restructuring the text to move towards this result. \n\nThanks for the suggestion. We have done this.\n\n> The interpolation in Eq. 7 only presents the mean-representation. There is no such discussion on the covariance.\n\nWe have revised the paper to include the covariance.\n\n> Notation for $p(x, x^+)$ in Eq. 2\n\nThanks for the suggestions here. We have revised the paper based on them.\n\n>  I suggest making Assumption 2 a proposition and writing an assumption around this that discusses approximation error\n\nThanks for the suggestion – we have done this.\n\n> Clarify which entries are zero in Lemma 3\n\nThanks for the suggestion – we have done this.\n\n> Missing proof of Lemma 3, no reference to appendix.\n\nThe proof of Lemma 3 is in Appendix A.4. We have added a link to this Appendix section right after the statement of Lemma 3.\n\n> Line 166 typo: 'that that'\n\nWe have fixed this.\n\n> in the first sentence line 187 say $\\psi(x)$ and $\\phi(x) = A \\psi(x)$\n\nThanks for the suggestion – we have done this.\n\n> Line 215, refer back to Eq.3 for the definition of $c$\n\nWe have done this.\n\n> Line 231, proof of lemma 2 is in appendix A.3, not A.2.\n\nWe have fixed this.\n\n> The three examples in 4.3 for lemma 2 are nice to have for an unfamiliar audience, but I don't see how this contributes to the result since $A$ is learned, and will likely never satisfy trivial solutions.\n\nWhen the matrix $A$ is learned on the datasets used in the paper, we empirically observe that it is often close to a rotation matrix (all eigenvalues are close to one). Thus, we believe that the assumption that $A$ is a rotation matrix (Examples 2 and 3) closely resembles practical settings. One can prove that $A$ will be an identity matrix if and only iff the dynamics are reversible (i.e., $p(x^+ \\mid x) = p(x \\mid x^+))$; thus, Example 1 is applicable when users work with reversible dynamics (e.g., a diffusion process with no drift).\n\n> Line 268, the interpolation result is nice, but the linear interpolation insight is not particularly profound\n\nWe have revised this sentence to de-emphasize this result.\n\n> The start and end symbols in Figure 4 are too small to see and overlap with the black dots too much, even at 500% zoom.\n\nWe have increased the size of these symbols.\n\n> Figures 5, 6 are not chronologically ordered/ numbered (6 appears before 5).\n\nWe have fixed this.\n\n> the authors could have spent more space on discussing the limitations of their method\n\nWe will do this.\n\n**References**\n\nOliveira, P. (2006). Interpolation of signals with missing data using PCA.\n\nZhang, et al. (2023). Universal visual decomposer: Long-horizon manipulation made easy", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lBTiXYpLyH", "reviewer": "Reviewer_GBML", "summary": "This paper presents a variant of contrastive learning to time series data. Contrastive learning has been used in many areas but the insight from this paper is joint distribution of representations is also Gaussian. In addition, the experimental results show that their theory can be applied to tasks up to 46 dimensions.", "questions": "1. As for section 5 Numerical Simulation, is it possible for the authors to use real datasets? For example, the authors can collect some datasets from the Traffic and Stock areas and evaluate the solution mentioned in this paper over them.\n2. In addition, I think efficiency is also important for high-dimensional tasks. Could the authors show the running time of each experiment?\n\nMinor:\n1. As for citation, the template of NeurIPS is [10, 12] instead of (10;12). I am not sure whether you have changed the template, but it would be great if you could fix it.", "limitations": "I think the authors have adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The problem studied in this paper is fundamental.\n2. This paper provides a strong theoretical analysis.\n3. The writing of this paper is clear. It is easy for me to follow the main idea of this paper.", "weaknesses": "1. Using contrastive learning to address various problems is not novel. However, this paper provides some new insights such as representations learned via temporal contrastive learning following a Gauss-Markov chain.\n2. The evaluation section is too simple, and it is possible to add more evaluation. Since it is an ML conference, more experimental results can make the paper more solidated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for the detailed review and for the suggestions for improving the work. We have run an additional experiment on a Stock price datasets, and revised the paper to include the running time for each experiment. **Together with the responses below, does this fully address the reviewer's concerns about the paper?**\n\nKind regards,\n\nThe authors.\n\n> is it possible for the authors to use real datasets? For example, the authors can collect some datasets from the Traffic and Stock areas and evaluate the solution mentioned in this paper over them.\n\nAs suggested by the reviewer, we ran an additional experiment on a Stock dataset. Data are the opening prices for the 500 stocks in the S\\&P 500, over a four year window. We remove 30 stocks that are missing data, so the resulting data is 470-dimensional. For evaluation, we choose a 100 day window from a validation set, and use Lemma 3 to perform ``inpainting'', predicting the intermediate stock prices given the first and last stock price (see Figure R2 in the rebuttal PDF). While we do not claim that this is a state-of-the-art model for stock prediction, this experiment demonstrates another potential application of our theoretical results. \n\n> Could the authors show the running time of each experiment?\n\nFor the datasets considered in this paper, the training time for temporal contrastive learning was less than 5 minutes per dataset. We will add the exact runtime for experiment to the final paper.\n\n> Using contrastive learning to address various problems is not novel.\n\nWe agree that prior work has used contrastive learning to address a wide range of problems. Our paper uses theoretical analysis to unlock a new use cases (e.g., the planning capability demonstrated in Fig 5). We will revise the paper to note that prior work has used contrastive learning to address several other downstream problems.\n\n> As for citation, the template of NeurIPS is [10, 12] instead of (10;12).\n\nWe have fixed this. Thanks for catching this!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers,

The attached rebuttal PDF contains the results of new experiments, which demonstrate stock prediction as a downstream application (Fig. R1) and which compare to a new autoencoder baseline (Fig. R2). Responses to each reviewer individually can be found in separate comments below.

Kind regards,

The authors.

Meta-review: While the paper's scores alone are borderline, the consensus I gather from reading reviews is that the paper should be accepted, with reviewers praising the paper's clear writing and various technical strengths, particularly after various clarifications given on rebuttal.
Decision: Accept (poster)