Paper: Course Correcting Koopman Representations (ICLR-2024)

Reviewer comment (Theory):
If I understand correctly, when $dim(z) > dim(x)$, the data points encode to a $dim(x)$ manifold embedded in the z-space. One interpretation is that the re-encoding takes latent points that have drifted away from the latent data manifold and puts them back on. However, if this were the case, re-encoding every step would have the best results.

Author response:
> “One interpretation is that the re-encoding takes latent points that have drifted away from this from the latent data manifold and puts them back on.”

Yes, this is a good way to describe issue (**2.**) discussed above. Indeed the results in **Figure 4(a)** (top left) serve as a confirmation of this. The results resemble the superposition of the phase plots of multiple linear dynamical systems, because this is exactly what is produced by linear latent dynamics and linear decoder without reencoding. The solutions are invalid over long horizons because of _overexploitation using a stale linearized representation_. Reencoding infers a new support and thereby updates the linearized representation. 

---

> “However, if this were the case, re-encoding every step would have the best results.”

Yes, we agree that this reprojection onto the data manifold is a good interpretation. However the reprojection is not perfect, and _introduces its own error_. Rolling out the dynamics in a regularized latent space for limited horizons is one way to mitigate the accumulation of such errors. In a way, reencoding, while not perfect, is a tool we use in moderation to address the issue of drift. This approach aligns with the findings of Mezic et al., who demonstrated that the Koopman operator can be locally extended within basins of attraction.

Reviewer comment (Theory):
I struggle to agree with the conclusions drawn by the example given in Appendix C. I view Appendix C as a special case of failure to specify good mappings. The claim that periodic reencoding solves the problem I find strange. If I understand correctly, the perturbations introduced by the decoder-encoder pass allow the latent state to jump between basins of attraction of each fixed point? Presumably these perturbations are just noise and an artefact of the (arbitrary) precision achieved by model training convergence.

Author response:
> “If I understand correctly, the perturbations introduced by the decoder-encoder pass allow the latent state to jump between basins of attraction of each fixed point?”
 
Yes, this is the idea presented in **Appendix C** and we discuss this in the "[General Response](https://openreview.net/forum?id=A18gWgc5mi&noteId=ddMyUvKB7n)." However, we disagree with the use of the term “perturbation” which implies a random change of support. The sparse Koopman Autoencoder model presented in **Appendix C** implicitly learns to assign a distinct support for each region of state space. Supports are not “randomly” assigned, but must correspond to the appropriate linearized representations corresponding to each region of the state space.

Reviewer comment (Experiments):
What is the benefit of periodic re-encoding, clamping latent representations to the data manifold, or adding noisy perturbations?

Author response:
> “what is the benefit of periodic re-encoding, clamping latent representations to the data manifold or adding noisy perturbations?”

Again, we disagree with the use of the term “noisy perturbation," please see our comment above. Periodic reencoding solves both issues: **1.** it ensures that the IVP principle is satisfied by construction, and **2.** it enables switching between representations corresponding to basins of attraction, i.e. proved in Lan & Mezic.

Reviewer comment (Experiments):
The authors include non-linear transitions as a baseline; while it is nice to have more baselines, as I understand, linear transitions are the only reason to use Koopman theory. As expected, the (strictly higher capacity) non-linear transitions outperform the linear transitions.

Author response:
> “...linear transitions is the only reason to use Koopman theory. As expected, the (strictly higher capacity) non-linear transition outperform the liner transitions.”

> “...this paper’s focus on the Koopman framework seems unnecessary.”

The primary motivation for studying Koopman autoencoders is their ability to ***learn rich, interpretable representations of nonlinear dynamical systems, a capability anchored in the linearity requirement of the Koopman theory***. Indeed, our paper presents several key results pertaining to this setting. Although our paper used periodic reencoding to overcome limitations related to linear transition dynamics (**Equations 8 & 9**, **Appendix C**) – it is nevertheless applicable to more general latent dynamics models that share some of the same limitations. 
Although we cannot use the closed form solution to linear dynamical systems to derive Equation 8, Equation 9 still holds for nonlinear latent dynamical models ($\mathit{K}$ would just be a nonlinear function of $x$). These models suffer from the same limitations as Koopman autoencoders -- they also violate the IVP without reencoding. 

We agree with the reviewer that models with nonlinear latent state dynamics are indeed 
less meaningful/interpretable than the linear latent dynamics, nevertheless we feel these experiments add value to the paper by broadening the applicability of our proposed method. 

Furthermore, there has been an increasing interest in the study of linear recurrent models recently, primarily due to their efficiency and the ability to run them in parallel using convolution. These models have demonstrated impressive results in tasks requiring long-range reasoning, as evidenced by the performance of State-Space Models. Notably, Koopman Autoencoders bear a close relation to linear recurrent models, a connection further explored in **Section E.1.2** of [arxiv.org/abs/2303.06349](https://arxiv.org/abs/2303.06349). A key contribution of our work is to scale Koopman Autoencoder to more challenging problems, by addressing the limitations pertaining to their particular setting.

Reviewer comment (Novelty):
Given the re-encoding is not specific to Koopman, and that Koopman (linear) transitions predictably perform worse than non-linear transitions, this paper’s focus on the Koopman framework seems unnecessary.

Author response:
> “...linear transitions is the only reason to use Koopman theory. As expected, the (strictly higher capacity) non-linear transition outperform the liner transitions.”

> “...this paper’s focus on the Koopman framework seems unnecessary.”

The primary motivation for studying Koopman autoencoders is their ability to ***learn rich, interpretable representations of nonlinear dynamical systems, a capability anchored in the linearity requirement of the Koopman theory***. Indeed, our paper presents several key results pertaining to this setting. Although our paper used periodic reencoding to overcome limitations related to linear transition dynamics (**Equations 8 & 9**, **Appendix C**) – it is nevertheless applicable to more general latent dynamics models that share some of the same limitations. 
Although we cannot use the closed form solution to linear dynamical systems to derive Equation 8, Equation 9 still holds for nonlinear latent dynamical models ($\mathit{K}$ would just be a nonlinear function of $x$). These models suffer from the same limitations as Koopman autoencoders -- they also violate the IVP without reencoding. 

We agree with the reviewer that models with nonlinear latent state dynamics are indeed 
less meaningful/interpretable than the linear latent dynamics, nevertheless we feel these experiments add value to the paper by broadening the applicability of our proposed method. 

Furthermore, there has been an increasing interest in the study of linear recurrent models recently, primarily due to their efficiency and the ability to run them in parallel using convolution. These models have demonstrated impressive results in tasks requiring long-range reasoning, as evidenced by the performance of State-Space Models. Notably, Koopman Autoencoders bear a close relation to linear recurrent models, a connection further explored in **Section E.1.2** of [arxiv.org/abs/2303.06349](https://arxiv.org/abs/2303.06349). A key contribution of our work is to scale Koopman Autoencoder to more challenging problems, by addressing the limitations pertaining to their particular setting.

Reviewer comment (Novelty):
There are many autoencoder + latent space transition models that vary mainly by application-specific architectural differences (e.g., pixels/graph/vectors require CNN/GCN/MLP encoder) and their training losses (L2, ELBO, Cross entropy, etc.), and a quick Google for "latent dynamics" and "deep kalman" yields many results, many that have linear latent transitions. Examples include:  
  - https://papers.nips.cc/paper_files/paper/2017/hash/7b7a53e239400a13bd6be6c91c4f6c4e-Abstract.html  
  - http://www.approximateinference.org/2015/accepted/KrishnanEtAl2015.pdf  
  - https://arxiv.org/abs/1811.04551

Author response:
> Existing Latent Space Transition Models 

The aim of the paper is to establish the issue of drift of latent states in Koopman Autoencoders and to present periodic reencoding as a simple and effective solution. We believe the effectiveness of periodic reencoding further sheds light on the implications of latent space trajectory generation, while readily being applicable to similar settings.

Reviewer comment (Writing):
Section 2, first paragraph does not help unfamiliar people understand Koopman theory or MDM (and familiar people do not need to be told). I would suggest simplifying this as the level of Koopman theory required for this paper can be explained very easily.

Author response:
> “Section 2, first paragraph does not help unfamiliar people understand Koopman theory or MDM (and familiar people do not need to be told). I would suggest dumbing this down as the level of Koopman theory required for this paper can be explained very easily.”

We agree. Thanks for your constructive feedback. We are posting the responses now to allow enough time for discussion regarding the main aspects of the paper. For the camera-ready version, we acknowledge that Section 2 needs a revision. We plan to undertake this revision, keeping in mind that it requires time.

Reviewer comment (Presentation):
Bibliography: many of the references have titles, names, and dates but are missing journal/conference information.

Author response:
> “Bibliography: many of the references have titles, names and dates but are missing journal/conference information”

We have fixed the bibliography to include the publication details.

Reviewer comment (Novelty):
Appendix E is a nice addition, yet it feels somewhat arbitrary to cite a handful of RNN papers when the time series modelling literature is vast and the autoencoder + latent dynamics literature is, I feel, more closely related to this work than RNNs.

Author response:
> “the autoencoder + latent dynamics literature is I feel is more closely related this work than RNNs.” 

Latent dynamics models are indeed closely related to recurrent neural networks, with the distinction between the two often being subtle. For instance, the _“world model”_ referenced in the work you cited ([arxiv.org/abs/1811.04551](https://arxiv.org/abs/1811.04551)), which is a latent dynamics model, is termed as “Recurrent State Space Model.” In light of this, we agree with your observation and **have updated our manuscript** to include a discussion on the relationship between these two approaches in the same Appendix section.

Reviewer comment (Reproducibility):
For each problem, can you state the dimensions of $x$ and $z$ in the main paper? (I couldn’t see them in Appendix B.)

Author response:
> “For each problem, can you state the dimensions of and in the main paper. (I couldn’t see them in Appendix B)” 

Information regarding latent space dimensions can be found in the concluding sentences of **Appendix B.1**. Additionally, we have updated the manuscript to include this information in the main text for easier accessibility.

Reviewer comment (Novelty):
Given linear latent transitions are worse and periodic re-encoding is more general than Koopman autoencoders, what is the justification for the authors to focus on Koopman representations? What is the benefit of the Koopman framework over any other autoencoder-based time series model?

Author response:
> “What is the benefit of the Koopman framework over any other autoencoder based time series model?”
 
We hypothesize that simpler latent transition mappings and decoder structures lead to **richer representations** of the dynamical system in question. To make accurate future predictions, the encoder needs to encapsulate and disentangle future state information into the current latent state as much as possible. Koopman theory shows that such linear mappings can indeed well approximate nonlinear systems, provided that they are of high dimensionality, ideally infinite-dimensional. The prescribed linear dynamical system is not only _easier to study_ but also offers greater **interpretability**. Additionally, it enables the **application of linear tools**, such as Linear Quadratic Regulator (LQR), for control purposes. Furthermore, linear dynamical systems, whether continuous or discrete, can be integrated/unrolled in time with **high efficiency and in parallel**. These factors collectively serve as strong motivators for conducting research on such systems.

Reviewer comment (Theory):
Regarding Equation 6: it appears the transition matrix is applying ½ of a forward time step and then ½ of a backward time step? I would understand if the transition matrix was made of many incremental time steps, e.g., $\textbf{K}=(I + \frac{\delta}{n}K)^n$ where $n =2$ (and consequently $\textbf{K}=exp(\delta K)$ as $n>\infty$).

Author response:
> “Equation 6: it appears the transition matrix is applying ½ of a forward time step and then ½ of a backward time step? I would understand if the transition matrix was made of many incremental time steps e.g. $\mathbf{K} = (I + \frac{\delta}{n} K)^n$ where $n = 2$ (and consequently $K=exp(\delta K)$ as $n > \infty$”

This is a standard discretization method used widely called blinear discretization, which S4 also uses. 

A simple derivation would be: $\quad e^{K \delta} = e^{K \delta /2} (e^{-K \delta /2})^{-1} \approx (I + K \delta /2)(I - K \delta /2)^{-1}.$

This is an instance of a standard method in the field of numerical resolution of ordinary differential equations (ODEs): the implicit midpoint rule. It is well documented in the literature, one reference being ["Solving Differential Equations 1" by Sylvert Paul Nørsett, Gerhard Wanner & Ernst Hairer, p204 Equation 7.4 & p205 Table 7.1]

Reviewer comment (Experiments):
What is the benefit of periodic re-encoding, clamping (high-dim) latent representations to the latent data manifold, or adding noisy perturbations?

Author response:
> “what is the benefit of periodic re-encoding, clamping latent representations to the data manifold or adding noisy perturbations?”

Again, we disagree with the use of the term “noisy perturbation," please see our comment above. Periodic reencoding solves both issues: **1.** it ensures that the IVP principle is satisfied by construction, and **2.** it enables switching between representations corresponding to basins of attraction, i.e. proved in Lan & Mezic.

Reviewer comment (Experiments):
The Periodic Reencoding works well especially with oscillator dynamics. It is likely related to the fact that those dynamics is periodic. It would be interesting to show how it works with continuous attractors.

Author response:
We show that our method can successfully capture the dynamics of a system characterized by chaotic attractors, such as Lorenz’63 and walking/running robots (D4RL). We think that an adaptive mechanism that adjusts the reencoding period is worth investigating for such systems.

Reviewer comment (Evaluation):
It's worthy discussing the "optimal" $\Delta t$ and its relationship to the dynamics. For example, Fig. 5 shows the best are the middle ones.

Author response:
Mezic et al. demonstrated that the Koopman operator can be locally extended within the basins of attraction of fixed points. As detailed in **Appendix C** of our paper, reencoding can function as a switch between the linear Koopman dynamics governing each region. Therefore, in theory, reencoding becomes necessary when trajectories traverse the boundaries between these basins of attraction. This suggests that the optimal reencoding period needs to be adjusted according to the rate at which we jump from one region to the other. 

A fascinating future research would be to develop a method to learn the optimal timing for reencoding, rather than relying on a fixed, periodic schedule. This approach could lead to a more adaptive mechanism. Toward this goal, we have been experimenting with training a reencoding policy network through Reinforcement Learning, which is still a work in progress. This network, conditioned on the latent state, outputs the decision of whether to reencode at each step. However, as of now, periodic reencoding still demonstrates superior performance compared to our implementation of this method.

Reviewer comment (Theory):
any underlying mechanism governing periodic re-encoding is not discussed. For example, are there deficiencies in the encoder or decoder that, if adjusted, remove the need for reencoding, or change the optimal period?

Author response:
> “any underlying mechanism governing periodic re-encoding is not discussed. For example, are there deficiencies in the encoder or decoder that, if adjusted, remove the need for reencoding, or change the optimal period?” 

> “what causes do we hypothesise for "long term drift"?”

One of the key issues with trajectory generation in latent space without reencoding stems from the lack of injectivity of the mappings from $z$-space into $x$-space. This lack of injectivity arises directly from the Koopman assumption that, $dim(z) > dim(x)$, a fundamental aspect of the setting. When reencoding is not employed, _trajectories_ in $z$-space are entirely decoded to _curves_ to $x$-space. Due to the non-injective nature of the mappings into $x$-space, the curves will cross over themselve, and this violates the IVP principle. This phenomenon occurs because multiple points along the $z$-trajectory are mapped to the same point in $x$ space, i.e. the crossover point. A simple diagram is [linked here](https://i.imgur.com/c7Hrv8d.png).

With reencoding, we are modeling the NLDS with a learned feedback (in $x$) which generates trajectories that do not cross by construction. The existence of the crossover point contradicts the fundamental principles of dynamical systems, suggesting that this method of inference is unsuitable. This deficiency, together with the inability to capture multi-attractor dynamics (**Appendix C**) is the source of what we refer to as 'long term drift' in the model's predictions. ***These issues cannot be overcome by architectural modifications to feedforward encoders/decoders.***


We establish that reencoding gives rise to valid solutions that do not violate the IVP principle; however, when used at every step, reencoding errors will compound and not follow the true trajectory of the system. The question of exactly why reencoding should be done only occasionally warrants further study, but we hypothesize that the reprojection is imperfect and can also accumulate errors. Thus we introduce periodic reencoding as a middle ground between no reencoding and reencoding at every step. We show empirically that this process generates the best predictions over long horizons.

Reviewer comment (Experiments):
how is the optimum period chosen? Since every step reencoding is worse in terms of performance than periodic reencoding, this is not a simple tradeoff in terms off costs

Author response:
> “how is the optimum period chosen? Since every step reencoding is worse in terms of performance than periodic reencoding, this is not a simple tradeoff in terms off costs” 

Our experiments empirically show that there is an optimal reencoding period, which should be tuned for every NLDS if we set aside computational cost and focus solely on prediction quality. We show that the method is relatively robust to the reencoding period as well (see error plots under varying reencoding schemes in the paper). One important aspect of our work is that it offers a spectrum from which the optimal reencoding period can be selected, according to the available computational budget and desired prediction quality. The computational budget may also vary based on the specific encoder/decoder used. The entries in the tables are based on the best reencoding period in terms of accuracy.

Reviewer comment (Experiments):
what causes do we hypothesise for "long term drift"? Perhaps during training the encoder does not learn to "use" all the latent space. Thus, there exist points in latent space with a decoding that is not properly trained. This is pure conjecture, but it would be beneficial for this paper to investigate these issues.

Author response:
> “any underlying mechanism governing periodic re-encoding is not discussed. For example, are there deficiencies in the encoder or decoder that, if adjusted, remove the need for reencoding, or change the optimal period?” 

> “what causes do we hypothesise for "long term drift"?”

One of the key issues with trajectory generation in latent space without reencoding stems from the lack of injectivity of the mappings from $z$-space into $x$-space. This lack of injectivity arises directly from the Koopman assumption that, $dim(z) > dim(x)$, a fundamental aspect of the setting. When reencoding is not employed, _trajectories_ in $z$-space are entirely decoded to _curves_ to $x$-space. Due to the non-injective nature of the mappings into $x$-space, the curves will cross over themselve, and this violates the IVP principle. This phenomenon occurs because multiple points along the $z$-trajectory are mapped to the same point in $x$ space, i.e. the crossover point. A simple diagram is [linked here](https://i.imgur.com/c7Hrv8d.png).

With reencoding, we are modeling the NLDS with a learned feedback (in $x$) which generates trajectories that do not cross by construction. The existence of the crossover point contradicts the fundamental principles of dynamical systems, suggesting that this method of inference is unsuitable. This deficiency, together with the inability to capture multi-attractor dynamics (**Appendix C**) is the source of what we refer to as 'long term drift' in the model's predictions. ***These issues cannot be overcome by architectural modifications to feedforward encoders/decoders.***


We establish that reencoding gives rise to valid solutions that do not violate the IVP principle; however, when used at every step, reencoding errors will compound and not follow the true trajectory of the system. The question of exactly why reencoding should be done only occasionally warrants further study, but we hypothesize that the reprojection is imperfect and can also accumulate errors. Thus we introduce periodic reencoding as a middle ground between no reencoding and reencoding at every step. We show empirically that this process generates the best predictions over long horizons.

Reviewer comment (Presentation):
text is not legible in some figures (Fig 3, 4, 5 font size too small, Figure 6 more extreme)

Author response:
> “text is not legible in some figures” 

Thank you for bringing this to our attention. We will increase the font size for the camera-ready version.

Reviewer comment (Novelty):
has equation (2) been used in previous papers? I see that Lusch et al. (2018) used something similar. It would be helpful to precisely clarify.

Author response:
> “has equation (2) been used in previous papers? I see that Lusch et al. (2018) used something similar. It would be helpful to precisely clarify.” 

We have updated the manuscript to mention previous works optimizing the same objective, i.e. Equation 2.

Reviewer comment (Theory):
I feel the theoretical benefits of re-encoding are not well explained. It appears that re-encoding should be most useful in eliminating undesirable local optimas; if the model is correctly specified, the AE objective should have the same global optima with or without re-encoding. It is thus not clear whether it is a principled solution to the switching dynamics issue, which is one of the two motivations for this work; if the dynamics cannot be (globally) linearized the model is misspecified in either case.

Author response:
> “It appears that re-encoding should be most useful in eliminating undesirable local optimas; if the model is correctly specified, the AE objective should have the same global optima with or without re-encoding.”


The main challenge that we aim to address via periodic reencoding, is to ***unroll the trained model over extended horizons***, i.e. longer than the sequence lengths that can be used at training time. If the objective was to predict future states for the same number of steps as in the training sequence length, then the problem is “correctly specified” and use of periodic reencoding would be unnecessary. This also necessitates the assumption that the model is equipped with a _nonlinear_ decoder with large capacity, which prevents the system from learning rich latent representations. In our work we analytically show that periodic reencoding is absolutely necessary, _even for predictions within the training sequence length_, when a linear decoder is used (**Section 3.2** and **Appendix C**). 


Periodic reencoding is crucial for accurately predicting future states ___beyond the scope of the training sequence___, even in the general setting with assumed nonlinear components. This necessity is substantiated both empirically and theoretically, as detailed in **Section 3.2** of our manuscript, where we show $x$-space trajectories, generated without reencoding, violate the Initial Value Problem setting. We kindly refer you to our post "[General Response](https://openreview.net/forum?id=A18gWgc5mi&noteId=ddMyUvKB7n)" for a more detailed explanation. 


As an alternative, one can choose to train the model with _nonlinear_ components over long sequences; however, this imposes hardware requirements and requires additional tricks to stabilize gradients. A number of papers cited in our work achieve this by engineering the latent dynamics. For example Unitary RNNs force the eigenvalues of the latent-to-latent transition to lie on the unit circle. These constraints reduce the expressivity of the model (arxiv.org/abs/2110.07238). We do not impose such constraints on the latent transition matrix. 


For all dynamical systems, we find training over horizons longer than 10 steps (and 100 steps for D4RL tasks) to be extremely unstable unless the parameters of $K$ are somehow constrained. However, through periodic reencoding we can unroll the model **up to 1000 steps stably and accurately**. We use a small training dataset of 50 trajectories of length 500 steps and use training sequences of length 10, however, we are still able to unroll the model successfully for 1000 steps – **500 steps longer than in training**. 

---

> “if the dynamics cannot be (globally) linearized the model is misspecified in either case.”


According to Koopman theory, the operator $K$ drives the Koopman embedding of the states forward _infinitesimally_ in time, given a freshly encoded point. In other words, the Koopman matrix is **spatially global**, as it is fixed/constant and applicable to all encoded points, however **temporally infinitesimal**. _Lan and Mezic show that the **Koopman operator can be applied locally rather than infinitesimally**, as it can be extended to the basin of attraction corresponding to the initially encoded point_. **Appendix C** illustrates how this idea can be implemented in practice, as sparse Koopman autoencoder. We use periodic reencoding to mitigate this issue, which in your review is refered to as "misspecification," all the while enjoying the benefits provided by Koopman -- rich representations, efficiency, interpretability, etc.

Reviewer comment (Experiments):
While the experiments demonstrate the benefits of re-encoding for autoencoding approaches, it is not clear why we should restrict to such approaches for control. There is also a general lack of comparison with baselines, e.g. with Mondal et al (2023).

Author response:
> “it is not clear why we should restrict to such approaches for control.”


In our work we aim to highlight the limitations of the Koopman framework and introduce periodic reencoding as a way to counter them. We do not claim that periodic reencoding is a replacement to other innovations, nonetheless our model/approach attains high prediction quality. We use D4RL as an standard benchmark that is widely used to allow such comparisons at the discretion of the reader. The sole motivation for conducting the semi-open-loop control experiment is to provide readers with a more tangible metric. In contrast to state mean squared error (MSE), we offer the normalized reward achieved by the agent as a more relevant measure to downstream applications (e.g. semi-open loop control). 

Furthermore, Mondal et al. (can be considered as concurrent work according to the guidelines) report future state prediction error over the horizon of 100 steps for D4RL tasks, which is also the sequence length used during training. The challenge that our method addresses is to ***go beyond the training sequence at inference time***, hence we report the error for **300 steps** – which is **still better/comparable to 100-step error of Mondal et al**.

Reviewer comment (Theory):
Regarding the sentence "This characteristic becomes especially significant when the dimensionality of the latent space is substantially larger than that of the original space, i.e., n >> d, ...", why is this case considered more important? While it's true that a mapping to a larger dimensional space is not surjective, as discussed in the paper, it seems that the lack of injectivity of the map is more critical. So, since a mapping to a smaller dimensional space is not injective, wouldn't the case of n << d be more problematic?

Author response:
> “why is this (n >> d) case considered more important?”


In theory, the Koopman operator is an infinite-dimensional mapping between latent codes. However, for practicality, Koopman Autoencoders are learned end-to-end by assuming a high-dimensional but finite latent space, i.e., $dim(z) \gg dim(x)$ or $n \gg d$. 

Trajectories generated in the latent space are valid, in the sense that they adhere to the initial value problem (IVP) and do not cross. This stems directly from the equation $z_{t+1} = K z_t$, which defines a (linear) dynamical system. However, when reencoding is not utilized during inference, and trajectories from the latent space are simply decoded to produce _curves_ in the original space, a discrepancy arises. Unlike the latent space trajectories, these curves can cross, thereby _violating the IVP property, at the crossover point_.

Due to the non-injective nature of the mapping from the latent space to the original space, _multiple_ points in the latent space can map to the _same $x$_, the crossover point, in the original space. This property is _unavoidable_ when $n > d$, i.e. mappings from higher dimensional to lower dimensional space cannot be injective. Nevertheless, even if $d > n$, the mapping is not necessarily injective and we must rely on statistical learning to capture this constraint.

Reviewer comment (Novelty):
When it comes to learning chaotic dynamics, what are the advancements offered by the proposed method compared to using RNNs with teacher forcing, as presented in the following papers? [1] F. Hess, Z. Monfared, M. Brenner, D. Durstewitz, Generalized Teacher Forcing for Learning Chaotic Dynamics, Proceedings of Machine Learning Research (ICML 2023), (2023). [2] J. Mikhaeil, Z. Monfared, D. Durstewitz, On the difficulty of learning chaotic dynamics with RNNs, Advances in Neural Information Processing Systems (NeurIPS), (2022).

Author response:
> “...proposed method compared to using RNNs with teacher forcing”


Thank you for the references; they are indeed relevant. Please see our post “[General Response](https://openreview.net/forum?id=A18gWgc5mi&noteId=ddMyUvKB7n)”. In summary, periodic reencoding can be used _in conjunction_ with many existing methods and _and should not be viewed as a competing method_. ***Periodic reencoding addresses fundamental limitations of modelling feedback processes with feed-forward functions -- this includes not satisfying the IVP property.***


From a high-level perspective, periodic reencoding can be used as an inference-time method for models with a single recurrent layer, i.e. the latent state dynamics, while [1] (concurrent and new to us) and [2] (discussed in **Appendix F**) studies are more concerned with the gradient behavior during training time applicable to all RNNs.  However, similar to [2], we also do use a “sparse” teacher-forcing regime during training time, as we optimize the loss over multiple future steps, given only the ground truth initial condition as input (see Figure 1). We reiterate that periodic reencoding is conceptually different from teacher forcing, as stated in the paper. Furthermore, [1] rectifies gradients by interpolating between the teacher forced and model-generated latent states. On the other hand, periodic reencoding, when employed at inference time, addresses a fundamental limitation of Koopman autoencoders, in regards to Koopman autoencoders. We show that our method can successfully model chaotic dynamics, Lorenz’63 and D4RL environments, all the while achieving this using a single linear recurrent layer, $\mathit{K}$.

Reviewer comment (Theory):
Regarding eq. (9), it appears that the Jacobian $J_{\psi}$ of $\psi$ is assumed to be a constant matrix. If this is the case, how can one ensure that it remains a fixed matrix? It may vary depending on the specific points at which it is calculated and could differ across different points.

Author response:
> “...the Jacobian is assumed to be a constant matrix”


The Jacobian varies as a function of $x$. We do not assume a constant Jacobian and we have updated our notation to reflect this properly. The only assumption that we draw conclusions from is that the RHS of Equation 9 is a function of $x$, forming an instant feedback loop in dynamics.

Reviewer comment (Experiments):
How effectively would the proposed method perform in capturing the dynamics of a multi-stable system with a stable fixed point and a periodic attractor, or possibly with two periodic attractors?

Author response:
> “dynamics of a multi-stable system” 

Please see our post “[General Response](https://openreview.net/forum?id=A18gWgc5mi&noteId=ddMyUvKB7n).” _**Appendix C** illustrates exactly such an example_ and demonstrates that reencoding is absolutely necessary to capture multi-attractor dynamics when using a linear decoder. This is more difficult to prove for a general decoder, but nevertheless the example conveys the intuition of what makes this problem difficult. Furthermore, our model demonstrates high-quality prediction performance on the low-/high-dimensional, controlled, and chaotic dynamical systems, covering a broad spectrum of dynamics.

Reviewer comment (Experiments):
How changes in the hyperparameters $\Delta t$ or $k$ can impact the results? Does an optimal value always exist for that?

Author response:
> “changes in the hyperparameters $\Delta t$ and $k$” 


In our experiments, longer training horizons, $k$, resulted in better performance, whereas shorter training sequences led to poorer outcomes. Therefore, we chose $k$ solely based on the stability of training and hardware constraints. $\Delta t$ is the duration between two reencoding operations at inference time. We show that there exists an optimal $\Delta t$, given that never/every-step reencoding schemes result in poor prediction quality.

Reviewer comment (Experiments):
Why do the authors use an MLP instead of K (e.g., on page 6)? How does the choice of activation function affect the performance of the MLP? Were other activation functions considered, and if so, how did they compare to the chosen function? Can the MLP be replaced with other types of neural networks, such as convolutional neural networks or recurrent neural networks? How would this affect the performance of the proposed framework?

Author response:
> “Why do the authors use an MLP instead of K?”

To clarify, we indeed conduct experiments using the K matrix for latent-to-latent mapping, and the results of these experiments are reported in all tables under the 'Koopman Autoencoder' section. The entries labeled 'Nonlinear Latent Dynamics' in **Table 1** pertain to the models equipped with a _wide MLP_, as the latent-to-latent mapping. Although periodic reencoding was initially motivated for models with linear latent transitions dynamics, our empirical findings demonstrate that periodic reencoding is also useful when using non-linear dynamics, which we believe further broadens the applicability of our work to more general models. 

Regarding activation functions, in our experiments with low-dimensional dynamical systems, ReLU achieved slightly better results than ELU. We attribute this to induced sparsity by ReLU, by the intuition from the example in **Appendix C**. We have not conducted experiments with various architectures for latent-to-latent mappings, but we have reasons to believe that periodic Reencoding would still enhance the prediction quality.