Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a methodology for predicting the equations of polynomial ODEs and obtaining embeddings of dynamical systems which can be applied to denoising data and classifying time series based on their physical properties. The authors propose a novel architecture based on CNNs to encode geometric features of vector fields, which is trained to minimize reconstruction losses. A FC layer predicts polynomial ODE coefficients, which are used for predicting equations that haven't been seen previously. They present results in synthetic datasets generated from polynomial ODEs, and utilize meteorological data to showcase their embedding space is meaningful. 
strength_and_weaknesses: Strengths:
- S1. Interesting and important problem. 
- S2. Recapitulation of relevant related work is very good.
- S3. The methodology is sound and non-trivial contribution.

Weaknesses:
- W1. The evaluation does not appear to be systematic. In neither the paper nor the appendix, the authors specify which dataset/ODE they are using for the examples of dynamically-informed denoising. It is possible that this is anecdotal evidence and not a legitimate claim for the paper. Additionally, I wonder why authors did not compare their method with other denoising methods and compute error metrics.
- W2. I found their method's design choices difficult to understand. There were some of them that I had to read several times from the appendix to the main paper. Other times, the information was simply not available. For example, why should we use the cubic expansion? Did you find this out experimentally?
- W3. The physics-informed term commonly refers to the physics-informed neural networks (PINNs), but authors use it with a significantly different meaning. It might be a good idea to think of another term or at least another title.
- W4. Readability would be benefited from putting clear examples, especially when addressing the mathematics in Section 3.

Questions:

- Q1. Can you please response to questions in W1?
- Q2. Please see questions stated in W2.
- Q3. Can you elaborate on why LASSO should be the only baseline for results in generalization to held-out systems? It is hard to judge if these are good results when compared to one simple baseline, and it’s not clear which were the reasons to exclude them from the evaluation to the several methods listed in the related work.
- Q4. From which literature did you obtain the physical properties of the underlying data (stability of fixed points, conservation of energy, and the incompressibility of flows)? Is this an exhaustive list?

clarity,_quality,_novelty_and_reproducibility: Although I am not familiar with literature on the specific challenge addressed by authors, I do believe they did a good job with their related work section and showing the novelties of their approach. This paper proposes a good method, but it is hard to follow on its most technical parts, and its evaluation does not seem convincing (for the reasons outlined above).
summary_of_the_review: The main concern I have with this paper is the lack of systematic evaluation in some sections. Although I have some concerns about the presentation, I hope authors add more details and make it more digestible. In my opinion, this paper is marginally over the threshold for acceptance. I would be willing to increase my score with the authors responding to my concerns on evaluation.

----
**After authors response:** I changed my decision to ACCEPT. See thread for details.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes phase2vec - a self-supervised framework for learning representations of dynamical systems in 2D. The main technical contribution is to use a 2D convolutional network for embedding dynamical system velocities, and formulating a reconstruction loss based on the data creation process to train the network weights. Experiments demonstrate that phase2vec can generalise to previously unseen dynamics equations, noisy systems, and real-world climate data, as well as serve as a downstream classifier for physical properties of dynamical systems.
strength_and_weaknesses: ### Strengths
+ The paper is generally well written and well motivated. The Introduction is readable by non-experts and contextualises the contributions well.
+ The proposed technique is straightforward and easy to understand. The main technical contribution - combining a 2D ConvNet and self-supervised loss applied to dynamical systems - is novel to the best of my knowledge. Phase2vec is an autoencoder for dynamical systems - which by itself isn't new - but the formulation of using a 2D ConvNet and self supervision signal is. (I would like to see the authors address the question of novelty during the rebuttal, see my questions below.)
+ The experimental evaluation seems thorough - phase2vec is probed for its ability to generalise outside its training distribution as well as on real world data. (But I would be keen to read other reviewers' opinion on the novelty of the technical contributions as well as the experimental rigour.)

### Weaknesses
- As the entire methodology and experiments are only geared towards dynamical systems in 2D Euclidean space, I would have liked to see this mentioned in the Abstract/Introduction. The only place where this is mentioned is in the Conclusion. (I realise this can be opinionated.)
- I found the Methodology and Related Work sections to have minor clarity issues which could be addressed by re-writing. I would like the authors to better contextualise the novelty of the proposed approach.

### Questions and Feedback
1. I appreciate the Related Work headers. While the section does a good job at surveying the literature, I was unsure how this paper, its methodology, and its experiments fit into each of those headers. Could the authors contextualise how their work is situated compared to what has been done. This may help to explicitly identify the main novelty of the paper w.r.t. existing literature. As a concrete example, there is a paragraph on vector-field-based approaches - but does phase2vec use them - if yes, how? 
2. Along this line of contextualising the novelty w.r.t. related work, could the authors clarify whether the proposed self supervised reconstruction loss is original? Are there similar works which encode observations to a latent dimension and run dynamics there?
3. Section 3.1. describes the 2 x n x n input array. How is the size n chosen? Does n need to be fixed? Assuming not, could the authors comment on how sensitive phase2vec is to changes in the input size?
5. Currently, the 2D ConvNet operates on a 2D grid array where each pixel/entry is a 2D velocity vector. Has phase2vec made any assumptions about the symmetries that may act on this vector space? E.g. is it realistic to assume that the velocity vectors should be rotationally equivariant (i.e. if the system is rotated, they would rotate along with it)? How would additional symmetry constraints impact the design of phase2vec and the underlying ConvNet + self-supervised loss function used?

### Minor Comments
- Along these lines, in Figure 1, it may be useful to list the dimensionality of the intermediate tensors. Just to double check my understanding: the input is 2 x n x n, it goes through a series of convolutions, the final prediction is 2 x 10? 
- I found the paragraph under equation 2 to be unclear. I assume it describes the basis functions? What is the multi-index? Why cubic degree? Would the resulting basis be: $1, x_1, x_2, x_1^2, x_2^2, x_1 x_2, x_1^2 x_2, x_1 x_2^2, x_1^3, x_2^3$?
clarity,_quality,_novelty_and_reproducibility: **Clarity and quality:** the paper is generally well written and well motivated. The presentation is of high quality. 

**Novelty:** the main technical contribution of phase2vec - combining a 2D ConvNet and self-supervised loss applied to dynamical systems - is novel to the best of my knowledge. 

**Reproducibility:** No code is provided, which can limit follow-up work. However, datasets and experiments are described in some detail in the appendix.

As previously stated, I will be monitoring other reviewers' opinion on the novelty of the technical contributions as well as the experimental rigour.
summary_of_the_review: I currently recommend acceptance. Overall, the paper is well written, the proposed phase2vec method is straightforward, and the experimental evaluation demonstrates its usefulness across several scenarios. I would like the authors to better contextualise the novelty of the proposed approach in the rebuttal.

---

**Post-rebuttal:** Thank you to the authors for their rebuttal and revised manuscript. Based on the rebuttal as well as the other reviewers' summary, I would currently like to keep my original score/assessment.

**Post-discussion:** Apologies for the late action. Based on revisiting the discussion and the authors releasing open source code (I have not tried running it, but it looks modular, well thought out, and extensible), I have upgraded my score. I think the revised manuscript is an improvement over the original and addresses reviewers' concerns. I would like to re-iterate my moderate confidence.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The manuscript discuss how to encode 2D phase space images to shorter vectors. The embedding can be decoded to reveal the governing equations, recover simple, sparse models or de-noise the input preserving the dynamical relations. As the phase portrait is so fundamental in describing the dynamics, it is not a surprise the physical features like energy conversation, etc can be decoded out, as there are very visible in the phase space image.

The proposed pipeline feeds the phase portrait, through a bottleneck, to polynomials that can be used to reconstruct the phase plot through solving the decoded equation. A consistency error is set between the original and newly generated phase plots and the sparsity loss is set to keep the equations as simple as possible.

The training data is an ensemble of polynomial ODEs in such a way that the form of equation where the model was tested were removed from the training data - to check the generalisation capability. The results were compared to a traditional per-equation fitting method.

The authors demonstrate the encodings with an analysis of atmosphere.
strength_and_weaknesses: Phase space is a traditional way to understand what is happening in dynamical systems. This is a combined in a nice way to a traditional deep learning image classification pipeline, using the strengths of both the physical domain and the machine learning domain.  The idea of using equation terms as "words" in the encoding is excellent, as the decoding through solving the resulting sparse differential equation has a real chance of being the optimally sparse representation of the system.

Here lies also the weakness of the system - as small changes to a differential equation terms, can lead to big changes in the phase space, so the encoder may not be very stable. Restraining to polynomial functions of low degree may help on that, but at the same time they system becomes less able to encode such complicated behaviour. Would there be some Lipschitz condition based reasoning that would provide smoothness of the phase space transformation with respect to the changes of polynomial weightings or is the de-noising training with digital ensembles able to find stable encodings and move away for highly sensitive combinations? Or are the described normalization procedures able o solve this?
clarity,_quality,_novelty_and_reproducibility: The paper is very well written, to my knowledge original, clear and concise. Appendices give enough information for reproducibilty.
summary_of_the_review: The paper is valuable contribution to explainability research, where experimental data can be encoded in a physically meaningful manner that supports common intuition.  The network architecture is a variation of an auto-encoder, where part of the generator is replaced with a world model. I would really be interested seeing this connected to a control problem, where the intuition gained in the encoding is used to control the physical processes....  So, this is not only about seen that, done that, but also opens new ways for machine learning based, explainable, control systems, besides the more straightforward venue to dimensions greater than two.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a method similar to word2vec but applied to dynamic systems. The approach considers the phase space as an image, from which we can extract a latent representation summarizing the characteristics of the dynamics.
strength_and_weaknesses: The paper introduces an interesting method for the study of dynamical systems. One of the key points of the approach is the ability to generalize to different dynamical systems without requiring re-training, which is not the case for existing methods. The network architecture seems reasonable, as does the design of the criterion.

However, I believe the following points are debatable:

- Embeddings are learned using a phase field reconstruction task, through a sparse regression to the coefficients of the dynamic equation. This has several limitations in my opinion:
    - This requires the supervision of a relatively complete dynamics map, which is not easy to access in many situations. However, the authors show that their method is sufficiently robust to withstand failures and errors in the phase field.
    - Predictions are limited to the structure of the function dictionary used $\Phi_\alpha$. Since this contains only polynomial structures, a large number of dynamical systems become out of reach. Can the authors discuss this limitation in a more general framework?

- The use of convolution greatly limits the number of systems that can be studied, since the dynamics must be of dimension 2. Convolution can possibly be extended to higher dimensions, but this becomes non-trivial. This is for me one of the most negative points of the approach.

- The experiments are for me rather unconvincing, in particular the reconstruction of the dynamics. Of the 10 systems studied, LASSO greatly outperforms Phase2Vec on 4 systems. When Phase2vec dominates, the gap with LASSO is smaller. However, I acknowledge that Phase2Vec do not requires training to infer on unseen dynamics, while it is not the case for LASSO. The denoising and masking experiments are very interesting, but quite unrealistic for me. Would it be more relevant to generate the phase field from a set of trajectories whose initial conditions will be chosen at random? This will make the phase field more representative of what can be obtained in practice.

- I would also be curious to see how the network behaves beyond the training set, i.e. outside the spatial limits of the phase plane used to estimate the embedding.

MINOR
- I found experiments on real world data a bit hard to read.
- 3.1: "at a discrete LOCATION"
clarity,_quality,_novelty_and_reproducibility: The paper is well written (with a minor exception for the last section), seems interesting and novel yet not very efficient, and limited to rather simple dynamical systems. However, reproducibility is not straightforward. Extensive description of the model should be added in the appendix. Code release would have been appreciated too.
summary_of_the_review: This paper introduces an interesting idea, but too limited on the number of possible applications. It is, however, an important line of research, which deserves to be taken into account.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper introduces a new method for embedding dynamical systems and predicting ODE equations, which can be applied to denoising and classification. Reviewers agree that this is an interesting and important problem, and that the method is sound and well-motivated. Most reviewers agreed that the experiments are thorough and show promising results. The primary concern, raised by multiple reviewers, is that the method is only applied to 2D systems, and can only ever work for rather low-dimensional systems in its current form. In my view the authors have done a decent job motivating the importance of 2D systems, and have provided a 3D implementation as well. As such I recommend the paper be accepted to ICLR.
justification_for_why_not_higher_score: Method is only applicable to 2D systems
justification_for_why_not_lower_score: I found this to be an interesting and innovative paper. Reviewers mostly agree it is of high quality (8,8,8,5), with the concerns raised by (5) seemingly minor
Decision: Accept: notable-top-25%