Paper: PICL: Incorporating Coarse-Grained Data and Physics Information for Superior Physical Systems Modeling (ICLR-2024)

Reviewer comment (Writing):
It is not very clear what the proposed framework is trying to achieve. The authors quotes "predictive ability" constantly but the prediction task is nowhere explicitly defined. I am inferring two tasks: (a) given coarse-grained state at time $t$, predict coarse-grained state at time $t+1$ and (b) given coarse-grained state, predict the corresponding fine-grained state. Task (a) inevitably involves incomplete information, which is not talked about and task (b) seems to be for a single step? (see questions below)

Author response:
> **Question about what does the prediction task mean and solution for incomplete information.**

**A:** Thank you for your question.  

- Your inference 'a' is correct. In our paper, "predictive ability" refers to the capacity to predict the subsequent coarse-grained observations from the current coarse-grained observation. In light of your question, we have now included a clear definition and explanation of "predictive ability" in both the Abstract and the concluding paragraph of the Introduction. 
- For your mentioned incomplete information, we use coarse-grained observations from previous steps to estimate the current fine-grained state inspired by the finite difference method, to compensate incomplete information caused by coarse-grained observed data. We added the description discussion of our method in Section 4.2.1.
-  Furthermore, for the impact of historical data length, we already did the ablation study and analyzed it in the third paragraph of Section 5.5.

Reviewer comment (Experiments):
The reason why the transition module is needed in the first place is unclear when there is already a closed expression for it from physics? It does not seem to provide benefits in terms of computational cost. If it represents approximate knowledge, the experiments do not demonstrate the value and limitations (compare against no physics knowledge at all and study how much error causes the framework to break down).

Author response:
> **Question about why the transition module is required.**

**A:** Thank you for your question. We also considered this aspect at the inception of our method's design, primarily for three reasons: 

- Firstly, numerous related studies, such as FNO [1], MAgNet [2], DINo [3], NeuralStagger [4], and others, focus on approximating numerical solvers using neural networks. Consequently, we decided to follow the ideas of these works, opting for neural network approximations rather than directly implementing numerical solvers.
- Secondly, in response to your question, we have conducted an additional experiment to compare the computational costs of the neural network in the transition module versus a numerical solver, which demonstrates the value of the transition module. This comparative study is detailed in the Appendix G. The findings are as follows: The time taken for a single computation using a numerical solver is more than five times that of a step inference with a neural network. Furthermore, as the resolution increases, the computational cost of using a neural network does not significantly change, as only the input and output layers are affected, with small changes in the hidden layers. These results indicate that, in terms of computational cost, whether in training or inference, employing a neural network in the transition module offers considerable advantages.
- Thirdly, employing a differentiable neural network is important for the future expansion of our proposed method. This approach may be beneficial for addressing control problems and inverse problems in subsequent research.

[1] Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. and Anandkumar, A., 2020. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895.

[2] Boussif, O., Bengio, Y., Benabbou, L. and Assouline, D., 2022. MAgnet: Mesh agnostic neural PDE solver. Advances in Neural Information Processing Systems, 35, pp.31972-31985.

[3] Yin, Y., Kirchmeyer, M., Franceschi, J.Y., Rakotomamonjy, A. and Gallinari, P., 2022. Continuous pde dynamics forecasting with implicit neural representations. arXiv preprint arXiv:2209.14855.

[4] Huang, X., Shi, W., Meng, Q., Wang, Y., Gao, X., Zhang, J. and Liu, T.Y., 2023. NeuralStagger: accelerating physics-constrained neural PDE solver with spatial-temporal decomposition. arXiv preprint arXiv:2302.10255.

Reviewer comment (Writing):
The loss terms and metrics do not come with formulas and instead explained in dense text, making it difficult to follow.

Author response:
> **Suggestion in formulas of loss terms and metrics.**

**A:** Thank you for your insightful suggestions. We have made revisions accordingly. 

- In Section 4.2.2, we have added more clearly definition each component of the loss function, including data loss and physics loss, using precise mathematical formulations. 
- Furthermore, in the first paragraph of Section 5, we have introduced a formal definition for the reconstruction error $\epsilon$ using a mathematical equation, replacing the previous textual description.

Reviewer comment (Theory):
When the observations are extremely coarse-grained, they represent partial state and the map to fine-grained state is not one-to-one. How is this uncertainty accounted, especially given that the modules in the proposed framework are all deterministic in nature?

Author response:
> **Question about handling non-one-to-one mapping.** 

**A:** Thank you for your comments. I have clarified a few points regarding the non one-to-one mapping issue:

- Due to the partially observed nature of the problem, we do not assume a one-to-one mapping between the current state $u_t$ and the next state $u_{t+1}$.
- Instead, as described in Section 4.2.1, we use coarse-grained observations from previous steps to estimate the current fine-grained state. There are two interpretations of this: First, similar to PDEs, we can use the temporal derivative estimated from historical data to approximate the spatial derivative and reconstruct the current fine-grained state. Second, we assume a one-to-one mapping exists between a sequence of historical coarse-grained observations and the current fine-grained state.
- We performed an ablation study (third paragraph, Section 5.5) analyzing the impact of the length of historical data used. This provides some empirical analysis of the mapping between observations and states.
- As you pointed out, uncertainty-aware models like Bayesian neural networks and variational autoencoders are promising approaches for handling partial observations. I have added a discussion of these methods in Section 4.2.1.

Reviewer comment (Evaluation):
Does evaluation involve only a single step prediction forward in time? Have you studied the performance over multiple steps?

Author response:
> **Question about multi-step performance.** 

**A:** Thank you for your question. Please note we have already show the multi-step prediction tests and compared them with baselines in our main paper. Please refer to Sections 5.2, 5.3, and 5.4, with results illustrated in Figure 2. To avoid confusion, we have revised the first paragraph of Section 5 by following your suggestion.

Reviewer comment (Writing):
Broken sentence in section 4.1 below equation (3) - "the sequentially used transition module roles the prediction process..."

Author response:
> **Suggestion about sentence expression.** 

**A:** Thank you for your valuable suggestion. We have revised Section 4.2.1 to more accurately describe the transition module's role in the prediction phase. To be more clear, this sentence means the transition module can achieve the forecasting function. We design a neural network that inputs the current fine-grained state  $\hat{u}\_{t}$  to forecast the subsequent fine-grained state  $\hat{u}\_{t+1}$.

Reviewer comment (Novelty):
Overall, the novelty in the paper is somewhat limited and analyses of the drawbacks of the proposed finite-difference based physics encoding method have not been fully carried out. Specifically, discussions regarding where the proposed method might lack, how fine-grained data can be incorporated (when available) will be helpful additions to the narrative.

Author response:
**A:** Thanks for your question.

- For the limitation of our proposed method: In Appendix D of our paper, we discuss the limitations of our work.
- The first one is the compatibility of our proposed framework and the specific neural network architecture. We have conducted addtional experiments and made a detailed discussion in the 11st paragraph of Appendix B.6.
- The second one is how to incropreate the avalable fine-grained data. We have conducted addtional experments and made a detailed discussion in the last paragraph of Appendix B.6. You may also check the following Question and Answer.


> **Questions about the usage of fine-grained data.**

**A:** Thanks for your question.

- For aspect of how fine-grained data can be incorporated into our framework:
  - Our current work is primarily focused on scenarios where only coarse-grained data is available, aligning with a significant portion of both simulation or real-world applications. 
  - We conjecture that if we assume the fine-grained data is available, the predictive performance will be better.
  - To verify the above conjecture, we conducted an additional experiment incorporating fine-grained data. We detailed the experiment and its results in last paragraph of Appendix B.6. In this experiment, fine-grained data was used as the label for the encoding module, aiming to train the module to reconstruct a more accurate learnable fine-grained state. Our findings show that both the baseline FNO* (without physics loss) and our proposed PICL benefit significantly from the inclusion of fine-grained data, with improvements of over 38% and 12% respectively.
  - Moreover, these results also highlight an important aspect of our framework: the ability to use the known physics information as a substitute for fine-grained data in training the encoding module, especially in scenarios where such data is unavailable. The smaller improvement observed in PICL compared to the FNO*, when fine-grained data is available, suggests that PICL can effectively compensate for the absence of fine-grained data to learn the state. Such discussion is also placed in Appendix B.6.

Reviewer comment (Evaluation):
Some results indicate that baselines outperform the proposed method. E.g., Table 1 NSWE indicate that PINO* has lower re-construction errors. Why is this?

Author response:
**A:** Thank you for pointing this issue. 

- For $\epsilon$ of PINO* outperforms PICL in Table 1, based on our experimental log data, we realized that there is a typo. The value should be 3.18E-1 instead of 3.18E-3. We have re-run the experiment multiple times, confirming that the magnitude is indeed in the E-1 range. We have fixed it in the table and extend our gratitude for your attention to this clerical error in our paper.
- To avoid you from being confused. In Table 1, the PIDL's $\epsilon$ for LSWE performs better than PICL. We have explained it in Section 5.3, before. Even though the $\epsilon$ value in PIDL is slightly lower, this approach concentrates on training using only the physics loss, overlooking the importance of data constraints in predictions. Consequently, PICL has been developed to trade off between data and physics constraints, thereby improving the model's predictive ability. We consider this current observation to be normal, as our method aims to enhance the model's predictive ability using physical information in coarse-grained data, which is reflected in the $\mathcal{L}_{d}$ in our paper. The $\epsilon$ is only a metric to show the accuracy of the reconstructed fine-grained state, a step in our process, but not the end goal of our work.

Reviewer comment (Writing):
The paper methodology is hard to understand and needs to be significantly improved. The reviewer feels the entire methodology can be explained in 1–2 paragraphs (half a page) but is needlessly convoluted and interspersed with details making it hard to get a high-level idea.

Author response:
**A:** Thank you for your valuable suggestion, which has significantly contributed to the clarity and readability of our paper. We have revised Section 4 accordingly :

- Firstly, we have succinctly introduced our concept and methodology in Section 4.1, using clear and easily understandable language to provide an overall view. This approach ensures that readers can quickly grasp the essence of our work.
- Secondly, for those readers interested in the technical specifics, we delve deeper into the details of our method in the subsequent Section 4.2.

Reviewer comment (Writing):
There are many ambiguous phrases/design decisions that have been made without explanation.

Author response:
> **The main paper lacks self-contained detail and currently relies heavily on the appendix in Section 4.**

- We have revised Sec. 4.2.2 to describe the details of the two-stage fine-tuning period.
- We have revised Sec. 4.2.1 to introduce the definition and details of the hard-encoding operation in the main paper.
- We have also revised other parts, including the details of the physics loss, so that the description of the proposed model can be easier to understand without the appendix.

Reviewer comment (Writing):
What does “hard encoding” $\tilde{o}$ into the corresponding $\hat{u}_t$ mean?

Author response:
**A:** Thank you for your question. 

- To address your concerns, we have clarified the concepts of down-sampling and hard encoding in Appendix B.5 of our paper, ensuring a more explicit explanation. Your understanding i is on point, and we will respond to question ii based on this understanding.
- We think this question as closely related to the second aspect of your initial comment regarding "Questions about the usage of fine-grained data." As mentioned previously, our current focus is on scenarios where physics information is used to enhance the model trained solely with coarse-grained data. In line with your suggestion, we extended an experiment involving high-resolution (fine-grained) data, and put it in last paragraph of Appendix B.6. 
- This experiment confirmed that high-resolution data, when available, can indeed enhance the model's predictive capabilities. This is because if such high-resolution data is used as a label, the encoding module can reconstruct a more accurate state, which facilitates the prediction process in the transition period.

Reviewer comment (Writing):
Does “hard encoding” mean that assuming the low-res data was n/2 × h/2 and high-res data was n × h, that every 4th pixel in the high-res data would have the corresponding $\tilde{o}$ value? Or does it mean something else?

Author response:
**A:** Thank you for your question. 

- To address your concerns, we have clarified the concepts of down-sampling and hard encoding in Appendix B.5 of our paper, ensuring a more explicit explanation. Your understanding i is on point, and we will respond to question ii based on this understanding.
- We think this question as closely related to the second aspect of your initial comment regarding "Questions about the usage of fine-grained data." As mentioned previously, our current focus is on scenarios where physics information is used to enhance the model trained solely with coarse-grained data. In line with your suggestion, we extended an experiment involving high-resolution (fine-grained) data, and put it in last paragraph of Appendix B.6. 
- This experiment confirmed that high-resolution data, when available, can indeed enhance the model's predictive capabilities. This is because if such high-resolution data is used as a label, the encoding module can reconstruct a more accurate state, which facilitates the prediction process in the transition period.

Reviewer comment (Theory):
If “hard encoding” means the same as Point 7, would this design decision not overtly couple the high-res and low-res solutions? How might the high-res solution significantly improve upon the low-res solution with this constraint?

Author response:
**A:** Thank you for your question. 

- To address your concerns, we have clarified the concepts of down-sampling and hard encoding in Appendix B.5 of our paper, ensuring a more explicit explanation. Your understanding i is on point, and we will respond to question ii based on this understanding.
- We think this question as closely related to the second aspect of your initial comment regarding "Questions about the usage of fine-grained data." As mentioned previously, our current focus is on scenarios where physics information is used to enhance the model trained solely with coarse-grained data. In line with your suggestion, we extended an experiment involving high-resolution (fine-grained) data, and put it in last paragraph of Appendix B.6. 
- This experiment confirmed that high-resolution data, when available, can indeed enhance the model's predictive capabilities. This is because if such high-resolution data is used as a label, the encoding module can reconstruct a more accurate state, which facilitates the prediction process in the transition period.

Reviewer comment (Writing):
What exactly does “imitating the implementation of higher-order FD” mean in the context of the function of $f_\theta$ in equation (1)? Is there a finite difference (FD) operator that has been embedded into $f_\theta$? Or is there something special (i.e., some special input transformation) that has been applied to the inputs of $f_\theta$ that makes it “imitate” an FD operator?

Author response:
**A:** Thank you for pointing out the need for clearer language in our manuscript. We have revised the wording in the first paragraph of Section 4.2.1 to more precisely describe the meaning.

- Specifically, we have changed the term "imitating" to "inspired by." Here, we aim to provide intuition for why we incorporate the historical coarse-grained observation as input data. More concretely, when we leverage the finite difference (FD) operators, we actually use the temporal finite difference to calculate the spatial finite difference. This means the historical data used to calculate the temporal finite difference can help estimate the (spatial) fine-grained state at the current time. 
- Moreover, we performed an ablation study analyzing the impact of historical data length, which is discussed in the third paragraph of Section 5.5.

Reviewer comment (Reproducibility):
The method section lacks a clear explanation of how the physics loss is defined and how the 4th-order Runge-Kutta (RK) method is incorporated into the design.

Author response:
**A:** Thank you for your valuable feedback regarding our experiments.

- According to your question, we have added a more detailed explanation immediately following Eqn. 5 in Section 4.2.2. The enhancement includes an in-depth description of the physics loss and the introduction of the RK4 method are placed in Appendix B.2. 
- We first present the process of RK4, which can be considered as a function $F(u_t,u_{t+1})=0$. By using $F$, we can design two physics losses $\mathcal{L}\_{ep}(\theta)=F(u_t(\theta), u_{t+1}(\theta))^2$  for the encoding module and $ \mathcal{L}\_{tp}(\omega) = F(u_t, u_{t+1}(\omega))^2$ for the transition module. By minimizing the $\mathcal{L}\_{ep}$ and $\mathcal{L}\_{tp}$, the $u_t$ and $u\_{t+1}$ can be found, which means that the $u\_t$ and $u\_{t+1}$ can satisfy the RK4 time differentiation.

Reviewer comment (Experiments):
The experimental section only compares the method with Fourier Neural Operator (FNO)-based methods, omitting comparisons with other state-of-the-art neural operator methods, such as similar mesh-based methods like Magnet.

Author response:
**A:** Thank you for your suggestion. 

- Because FNO and PINO are common methods that many works about neural operator used as baselines, we previously compared our method with them.
- Thank you for mentioning MAgNet. We have done further literature review and found the work of three state-of-the-art neural operators, including MAgNet [1], DINo [2], and LatentNeuralPDEs [3], and cited them in the Related Work. Considering the limitations of computing resources, we selected the method LatentNeuralPDEs with the best performance as our baseline. 
- We have compared our proposed method with the official implementation of LatentNeuralPDEs. The details about LatentNeuralPDEs are described in Appendix C, and the comparative results are presented in Appendix E and Table 9. The findings indicate that the loss ($\mathcal{L}_{d}$) of LatentNeuralPDEs is larger than that of PICL w/o fine-tune and PICL with fine-tune by more than 81% and 91%, respectively.

[1] Boussif, O., Bengio, Y., Benabbou, L. and Assouline, D., 2022. MAgnet: Mesh agnostic neural PDE solver. Advances in Neural Information Processing Systems, 35, pp.31972-31985.

[2] Yin, Y., Kirchmeyer, M., Franceschi, J.Y., Rakotomamonjy, A. and Gallinari, P., 2022. Continuous pde dynamics forecasting with implicit neural representations. arXiv preprint arXiv:2209.14855.

[3] Iakovlev, V., Heinonen, M. and Lähdesmäki, H., 2023, November. Learning Space-Time Continuous Latent Neural PDEs from Partially Observed States. In Thirty-seventh Conference on Neural Information Processing Systems.

Reviewer comment (Experiments):
The original FNO paper tested model performance on different Partial Differential Equations (PDEs), including 1D cases (decay flow) and 2D cases (Navier-Stokes equations). It would be beneficial if the author also compared their method against these PDEs.

Author response:
**A:** Thank for your suggestion. We agree that additional experiments involving PDEs used in FNO are essential. Regarding your question about the 1D case decay flow, you might mean the Burgers flow experiment in FNO, as decay is a characteristic feature of the viscous Burgers equation? If not, please tell me what you mean.

- Based on your suggestion, we have conducted experiments on the Burgers Equation and the Navier-Stokes Equation following the settings in original FNO paper, and compared the results with FNO. The detailed experimental setup and results are presented in the Appendix F of our paper. 
- We summarize the results as follows: In the Burgers Equation experiments, the data loss ($\mathcal{L}_{d}$) of PICL with fine-tuning is marginally lower than that of FNO* and FNO. In the Navier-Stokes Equation experiment, which involves a more complex PDE, PICL with fine-tuning demonstrates significant improvement over both FNO* and FNO. These results are detailed in Table 10. This enhancement in performance shows the efficacy of our approach and its potential in handling intricate fluid dynamics problems.

Reviewer comment (Experiments):
The performance in zero-shot super-resolution (SR) cases is not addressed.

Author response:
**A:** Thank you for your insightful question. Initially, our work did not focus on addressing the issue of zero-shot super-resolution. Inspired by your questions, we have conducted additional experiments to assess the zero-shot super-resolution capabilities of our proposed framework. 


- The detail of experiment is descripted in the 13rd paragraph of Appendix B.6 named “How to address the zero-shot super-resolution cases?”. The conclusion is: The data presented in the Fig. 5 clearly demonstrates that the PICL with fine-tune consistently outperforms the baseline FNO*, regardless of the inference sizes of coarse-grained meshes. This indicates that the incorporation of physics information can significantly enhance the zero-shot super-resolution of the encoding module. This improvement is achieved by ensuring that the output adheres to the underlying physics equation, a crucial factor especially when fine-grained data is not available.
- Here is how we modified the network: The original UNet encoding module requires a consistent coarse-grained meshes between inference and training, making it difficult to complete the zero-shot super-resolution task. The goal of our work is to develop a training framework that is not limited to specific encoding module. Grateful for the inspiration provided by MAgNet, we apply the nearest neighbors interpolation as same as MAgNet in the encoding module. Then, by replacing the UNet to Transformer as encoding module, our proposed framework can address the zero-shot super-resolution cases.

Reviewer comment (Writing):
I do not quite understand the problem setting in the paper yet. What problems does the paper aim to tackle? To my knowledge, the prediction of coarse-grained observation is not a keen requirement for real world applications since simulation in real applications is mostly performed with high resolution data.

Author response:
**A:** Thank you for your question. To make our paper clearer and easier to follow, we have made the following adjustment to the paper:

- For the problem setting, the scientific problem we solved is how to apply physics information on coarse-grained data to improve predictive ability, whether in numerical simulations or real-world applications when we only have coarse-grained data. We revised the Abstract and the 2nd paragraph of the Introduction to make it clear.
- For its importance: 
  - In real-world scenarios, only coarse-grained data measured by sensors is available that is the importance of the problem setting.
  - Generally, this framework can be applied to any problem seeking to enhance model predictions using physics information on coarse-grained data, offering insights and potential solutions.

Reviewer comment (Experiments):
What is the difference between PICL and FNO* reported in the experiments? Are the loss functions used in training different? It is also unclear how much each term of the loss function has an impact to the performance of the proposed forward model.

Author response:
**A:** Thank you for your question.

- We have made revisions to Section 5.1, first paragraph, to provide a more detailed description to each baseline for better understanding. Here to clarify: To control variables, PICL and the FNO* baseline are very similar in terms of network structure. Specifically, the primary difference between PICL and FNO* is that FNO* omits the physics loss components from the PICL’s loss function, retaining only the data loss.
- Regarding the impact of each term in loss functions, our paper already discussed this in Section 5.5 Ablation Studies, specifically in the second paragraph. Our experimental findings indicate that including a physics loss in the loss function leads to better results compared to method such as FNO* without it. The model performs optimally when the physics loss coefficient, $\gamma$ = 1E-1. This is because a very low weight for physics loss makes it ineffective in the optimization process, while a very high weight causes the model to overly prioritize satisfying physical constraints at the expense of predictive accuracy. We appreciate you highlighting this issue, and we have made revisions to this section for greater clarity in describing the ablation study.

Reviewer comment (Experiments):
When the output resolution of $f_{\theta}$ increases, how much does it have an impact to the computational cost? How does it compare to the baselines?

Author response:
**A:** Thank you for raising this intriguing question.

- In response, we have added a group of experiment to Appendix B.6, specifically in the 9th paragraph, titled "When the output resolution of the encoder module increases, how much does it impact the computational cost?" 
- Our findings are as follows: During inference, as the resolution of the encoder module's output increases from $32\times32$ to $48\times48$, and then to $64\times64$, the computational time (s) for the baseline without physics loss increased by approximately 92% and 65%, respectively. In comparison, PICL saw an increase of about 83% and 69%. The growth in computational cost for both methods is quite similar. During training, the baseline without physics loss increased by about 277% and 79%, while PICL increased by approximately 238% and 48%. Although PICL's training time increased, its computational cost grew less than the baseline with the rise in resolution. In summary, when using PICL for inference, the computational cost considers only the time for inference. Hence, the efficiency of PICL is essentially comparable to the baseline without physics loss, as PICL does not require computation of physics loss during inference.

Reviewer comment (Presentation):
The paper does not seem to be well-organized and hard to follow.

Author response:
**A:** Thank you for pointing out the organizational aspects of our paper. We have undertaken comprehensive revisions across various sections, including the abstract, introduction, preliminaries, and methodology. 

Specifically:

- **Problem 1:** We appreciate your feedback and have reorganized the definitions of problems and operators in the preliminaries. The domain of the problem we want to solve is similar to existed works like FNO [1] and PINO [2], and it is also a prediction problem in the PDE-governed system. Therefore, we referred to their definition of the problem and revised the section Preliminaries to maintain consistency with these works. This includes a refined definition of the differential operator $P$, an enhanced discussion of their discrete representations, and the clear definition of down-sampling operation.
- **Problem 2:** We have revised Section 4.2.1 to more accurately describe the transition module's role in the prediction phase. To be more clear, this sentence means the transition module can achieve the forecasting function. We design a neural network that inputs the current fine\-grained state $\hat{u}\_{t}$ to forecast the subsequent fine-grained state $\hat{u}_{t+1}$.
- **Problem 3:** We have revised the last paragraph of the preliminaries for clarity and provided a detailed mathematical formulation and execution details of this operation in Appendix B.5. To be more clear, the down-sampling means we down-sample the state values in the fine-grained data $\tilde{u}$ and then rearrange them to a matrix that is the coarse-grained observation $\tilde{o}$.
- **Problem 4:** Thanks for your question. Before, we only show the training period in Figure 1. Now, we have added three figures to show the inference period, physics-tuning stage, and data-tuning stage in the paper, for the reader can easily understand. The figures of training period and inference period are placed in section 4.1, and the figures of physics-tuning stage and data-tuning stage in two-stage fine-tuning period are place in Appendix B.3.

Reviewer comment (Theory):
Definition in Section 3 is ambiguous. For instance, differential operator $P$ is not well-defined since Banach spaces $(A, U, V)$ is not mentioned to have differential structure.

Author response:
**A:** Thank you for pointing out the organizational aspects of our paper. We have undertaken comprehensive revisions across various sections, including the abstract, introduction, preliminaries, and methodology. 

Specifically:

- **Problem 1:** We appreciate your feedback and have reorganized the definitions of problems and operators in the preliminaries. The domain of the problem we want to solve is similar to existed works like FNO [1] and PINO [2], and it is also a prediction problem in the PDE-governed system. Therefore, we referred to their definition of the problem and revised the section Preliminaries to maintain consistency with these works. This includes a refined definition of the differential operator $P$, an enhanced discussion of their discrete representations, and the clear definition of down-sampling operation.

Reviewer comment (Writing):
Between equations (3) and (4), "Then, the sequentially used transition module roles the prediction process with the input $u^{t}$ to predict $u^{t+1}$".

Author response:
**A:** Thank you for pointing out the organizational aspects of our paper. We have undertaken comprehensive revisions across various sections, including the abstract, introduction, preliminaries, and methodology. 

Specifically:

- **Problem 2:** We have revised Section 4.2.1 to more accurately describe the transition module's role in the prediction phase. To be more clear, this sentence means the transition module can achieve the forecasting function. We design a neural network that inputs the current fine\-grained state $\hat{u}\_{t}$ to forecast the subsequent fine-grained state $\hat{u}_{t+1}$.

Reviewer comment (Reproducibility):
I could not find the implementation detail of down-sampling operator $\Phi$.

Author response:
**A:** Thank you for pointing out the organizational aspects of our paper. We have undertaken comprehensive revisions across various sections, including the abstract, introduction, preliminaries, and methodology. 

Specifically:

- **Problem 3:** We have revised the last paragraph of the preliminaries for clarity and provided a detailed mathematical formulation and execution details of this operation in Appendix B.5. To be more clear, the down-sampling means we down-sample the state values in the fine-grained data $\tilde{u}$ and then rearrange them to a matrix that is the coarse-grained observation $\tilde{o}$.

Reviewer comment (Presentation):
Figure 1 is very confusing since arrows corresponding to data flow in training and test phases are mixed up. Are physics loss $L_{ep}$ and $L_{tp}$ used in the unrolling phase?

Author response:
**A:** Thank you for pointing out the organizational aspects of our paper. We have undertaken comprehensive revisions across various sections, including the abstract, introduction, preliminaries, and methodology. 

Specifically:

- **Problem 4:** Thanks for your question. Before, we only show the training period in Figure 1. Now, we have added three figures to show the inference period, physics-tuning stage, and data-tuning stage in the paper, for the reader can easily understand. The figures of training period and inference period are placed in section 4.1, and the figures of physics-tuning stage and data-tuning stage in two-stage fine-tuning period are place in Appendix B.3.