Paper: Dynamic Mode Decomposition-inspired Autoencoders for Reduced-order Modeling and Control of PDEs : Theory and Design (ICLR-2024)

Reviewer comment (Experiments):
The paper only compared to DMD based methods, however, transformer based VAEs like VQ-VAEs and VQ-GANs have achieved SOTA performances on video data.

Author response:
We appreciate your suggestion to explore transformer-based VAEs for comparison in our study. However, application of these methods to controlled dynamical systems with dynamic control inputs is not straightforward and requires significant modification to the architecture. Recognizing this, we consider such models for our fluid flow example without any control inputs and investigate the prediction performance of VideoGPT (VQ-VAE + Transformer). Our experiment shows that VideoGPT generates accurate predictions in the short term but falls short in capturing long-term dynamics. We value your insightful suggestion and have included quantitative and qualitative results in the supplementary to encourage potential future research directions.

Reviewer comment (Experiments):
It would be very beneficial to see a comparison with these more data-driven methods.

Author response:
We appreciate your suggestion to explore transformer-based VAEs for comparison in our study. However, application of these methods to controlled dynamical systems with dynamic control inputs is not straightforward and requires significant modification to the architecture. Recognizing this, we consider such models for our fluid flow example without any control inputs and investigate the prediction performance of VideoGPT (VQ-VAE + Transformer). Our experiment shows that VideoGPT generates accurate predictions in the short term but falls short in capturing long-term dynamics. We value your insightful suggestion and have included quantitative and qualitative results in the supplementary to encourage potential future research directions.

Reviewer comment (Novelty):
The idea of replacing linear encoder and decoder with neural networks is simple. There is not much innovation in this paper.

Author response:
While we appreciate your assessment, it is important to note that the replacement of linear encoders and decoders with neural networks is not the primary focal point of our contribution. We acknowledge the existence of prior works exploring similar model configurations within the field. 

Our paper aims to establish a theoretical connection between the traditional linear approach and the emerging deep learning-based methods for the control of dynamical systems. There are several viable approaches to constructing and training an autoencoder-based ROM for PDE-driven dynamical systems, as demonstrated in recent literature. In our work, the architecture and loss functions are strategically formulated to forge a bridge between the conventional linear approach and DNN-based methods, specifically through the lens of a linear autoencoding representation of DMDc.

Reviewer comment (Experiments):
About the learning of the control. Since the control is of feedback type, can we use a simple supervised learning from the observed data instead of loss (15)?

Author response:
The data used for training the model encompass observations obtained under random control inputs. We assume that we do not have any trajectory data under an effective control policy. Without such data formulating supervised learning methods becomes challenging. We would appreciate additional elaboration on this comment, as it would help us understand the feasibility of employing supervised learning.

Reviewer comment (Writing):
Page 6 middle. In the integration, it should be t instead of t_i?

Author response:
Yes, it should be $t$. Thank you for pointing out the error.

Reviewer comment (Experiments):
Lack of ablation study. The paper mainly focuses on a new reduced-order approach (DeepROM) to model the system’s dynamics and adopts an existing method NI4C (Saha et al., 2021) to learn the control. Thus in the experiment, the original full-order NI4C should be compared, as an ablation study for DeepROM. However, in current experiments, only LQR controls are compared.

Author response:
**A1.** Thank you for highlighting the importance of an ablation study, specifically comparing the original full-order NI4C method with our proposed reduced-order approach. We recognize the significance of this comparison in evaluating the efficacy of our method. We have added a comparison with full-order NI4C in the revised paper for the fluid flow experiment. While the full-order model (FOM) trained for NI4C exhibits superior prediction accuracy for the initial steps, it experiences a rapid decline in accuracy over time. In contrast, DeepROM provides accurate predictions for a much longer period of time. On the other hand, FOM + NI4C shows better performance for the control task. However, it is crucial to highlight that the advantage of employing a reduced-order model over a full-order model in control learning primarily resides in reduced computational complexity without a substantial compromise in accuracy. Effective FLOP count (#E-FLOPs) per training or prediction step for FOM is significantly higher (over 150X) than that of DeepROM despite a similar parameter count. The substantial computational disparity arises due to the NI4C’s requirement for a continuous-time formulation of the model, necessitating numerical integration during both training and inference. 

|Model| |#Params |#FLOPs| NFE| #E-FLOPs|
|----------|----------|----------|----------|----------|----------|
|FOM ||76.93K| 22.34M| 42.83| 956.80M|
||Encoder| 23.11K| 1.17M | N/A| |
|DeepROM|ROM| 22.91K |34.00K |13.13 |6.11M
||Decoder| 23.11K| 4.49M |N/A|

NFE: Avg. number of function evaluations by the numerical integrator

#E-FLOPs(FOM) = #FLOPs(FOM) × NFE(FOM)

#E-FLOPs(DeepROM) = #FLOPs(Enc) + #FLOPs(Dec) + #FLOPs(ROM) × NFE(ROM)

Reviewer comment (Evaluation):
Insignificant performance improvement. The performance of the DeepROC on the vortex-shedding task seems to be worse than LQR baselines (Figure 3(b,c)). The overall idea of reduced-order control is not new, thus the performance improvement is an important measure of contribution.

Author response:
**A2.** We acknowledge your concern regarding the observed performance of DeepROC on the vortex-shedding task compared to LQR baselines. We recognize that performance improvement would undoubtedly strengthen the contribution.  We explicitly acknowledge this limitation in the conclusion section.

It is essential to clarify that our paper does not claim novelty in the general idea of reduced-order control. Instead, our focus was on establishing a theoretical connection between a traditional approach and the emerging deep learning-based methods for the control of dynamical systems.

Reviewer comment (Experiments):
Lack of higher dimensional and more complex datasets. In the abstract, the authors state that: "Controlling complex PDEs often necessitates dimensionality reduction for computational efficiency." However, the experiments are about 1D reaction-diffusion and 2D N-S (with low Re). Such systems can be learned and predicted efficiently via full-order neural networks, such as simple FNN, DeepONet, FNO, PINN, and their variants. More complex datasets, such as 2D/3D N-S with turbulence or shock are needed to show the necessity of reduced-order models.

Author response:
**A3.** We agree that several existing DNN-based full-order models can efficiently predict the evolution of systems that we considered in this paper. However, many of these models are not straightforwardly applicable to systems with dynamic control inputs. Furthermore, learning control through a DNN-based full-order model is computationally expensive. Existing methods either optimize for static control inputs using backpropagation through the learned model or employ model predictive control to learn dynamic inputs. With full-order models, such optimizations with long trajectory rollouts are computationally expensive. For example, the FLOP count for $T$-step rollout for FNO is much higher than that of a discrete-time version of DeepROM for a large $T$. 

|Model| |#Params |#FLOPs| #FLOPs for $T$-step rollout|
|----------|----------|----------|----------|----------|
|FNO||926.85K| 8.37M| $(8.37 \times T)$M| 
||Encoder| 23.11K| 1.17M ||
|DeepROM|ROM| 11.20K| 11.00K| $(5.66 + 0.011 \times T)$ M| 
||Decoder| 23.11K| 4.49M ||

Reviewer comment (Experiments):
How to choose tunable hyperparameter $\beta_{1,2,3}$?

Author response:
**A4.** $\beta_{1,2}$ were chosen from a set of values by evaluating the prediction and reconstruction losses on a validation set. $\beta_3$ is chosen from a set of values by monitoring the Lyapunov energy along the trajectory of the ROM, under the learned policy.

Reviewer comment (Experiments):
Does adding a decoder and a reconstruction loss for control $u$ affect performance?

Author response:
**A5.** The controller itself acts as a decoder. As mentioned in Appendix D.3 and shown in Fig. 9, one branch of the controller network learns space-dependent polynomial basis functions given the locations of the actuation nodes. The other branch maps the current reduced state to the coefficients for those basis functions. These coefficients can be considered as the latent code for control. However,  we did not observe any performance improvements by adding a reconstruction loss between these coefficients and the output of the control encoder.