LEARN TO ADAPT PARAMETRIC SOLVERS UNDER INCOMPLETE PHYSICS

Authors: armand kassaï koupaï
yuan  yin
patrick  gallinari
Venue: ICLR_cc_2024_Workshop_AI4DiffEqtnsInSci
Published: 2024-02-10 22:30:02

Abstract: Modelling physical systems when only partial knowledge of the physics is available is a recurrent problem in science. Within this context, we consider hybrid models that complement PDE solvers, providing incomplete physics information, with NN components for modelling dynamical systems. A critical challenge with this approach lies in generalising to unseen environments that share similar dynamics but have different physical contexts. To tackle this, we introduce a meta-learning strategy that captures context-specific variations inherent in each system, enhancing the model's adaptability to generalise to new PDE parameters and initial conditions. We emphasise the advantages of adaptation strategies compared to a pure empirical risk minimisation approach, the superiority of the solver-neural network combination over soft physics constraints, and the enhanced generalisation ability compared to alternative approaches

Main paper: 
 introduction: 
 Augmenting physical principles with data-driven methodologies has emerged as a promising approach for modelling dynamical systems and solving partial differential equations (PDEs). This can be achieved classically by incorporating physical constraints within a loss function (Raissi et al., 2019; Li et al., 2023) . However, this often leads to ill-posed optimisation problems, and incorporating hard constraints is usually more efficient. Hybrid systems that combine PDE solvers with NN components do exactly that. This idea has been explored, for example, for accelerating simulations by enhancing low-resolution solvers with neural networks (NNs) (Kochkov et al., 2021; Um et al., 2021) or for complementing partially known dynamics (Yin et al., 2021; Tathawadekar et al., 2023) . We consider here this hybrid setting for solving parametric PDEs, i.e. modelling physical systems subject to varying dynamics due to changes in equation coefficients, initial conditions (IC), boundary conditions (BC), and forcing terms.A key problem with models leveraging data-driven components is the generalisation issue: these models fail to generalise to unseen contexts or parameters. This is currently addressed through a classical empirical risk minimisation (ERM) practice, by sampling observations or simulations across the parameters distribution (Brandstetter et al., 2023; Takamoto et al., 2023) . However, given the complexity and the diversity of observable dynamics in physical systems together with limited data availability, these methods primarily interpolate in a small neighbourhood within the training parameters distribution, lacking generalisation to unseen conditions outside the training ones. Our claim is that empirical risk minimisation is not well-suited for modelling the complexity of physical dynamics. Instead, we advocate for an alternative approach that emphasises systems capable of rapidly adapting to novel environments. Our framework can be viewed as a meta-learning approach for rapidly adapting the data-driven components of a hybrid model. Our contributions are as follows:• We introduce an adaptation strategy for hybrid Physics-aware neural parametric PDE solvers providing enhanced extrapolation abilities compared to alternative approaches. • This allows low-cost adaptation to new situations with only a small number of observed data. • We evaluate the benefits of this approach across various datasets, encompassing both-in-distribution and out-of-distribution scenarios. • We compare this hard physical constrained approach to soft constraint incorporation. 
problem description and notations: 
 We consider general dynamics driven by equations of the form:EQUATIONwith u(x, t) the PDE solution at time t ∈ I ⊂ [0, ∞) at position x in space Ω, f ∈ F maps the PDE solution u to its temporal derivatives. Our objective is to model the temporal evolution of a deterministic spatio-temporal phenomenon. We posit the observation of similar phenomena occurring in diverse environments e, each leading to a distinct dynamics denoted f e . Specifically, we assume a dynamical system modelled by a parametric PDE, where the form of the PDE is shared across environments, but the parameter values vary among them. We also assume that the physics is partially known and provided as an explicit PDE. The dynamics equation for a given environment can thus be rewritten as:EQUATIONf e and r e respectively represents the known and unknown physics of environment e and G is a simple function combining the known and unknown components of the PDE.For training, we assume the observation of a set of trajectories for each environment e, each defined by an IC u 0 . We define E tr and E ad , respectively the set of environments used to train the model and to adapt to during inference. In each training environment, a small amount of N trajectories is available to train the model and composes a training set D e tr = {u i (x, t)} N i=1 . During inference, we adapt our model using only a single trajectory per environment denoted as D e ad . We define the MSE loss over training set D e tr (more details in Appendix C):L(θ, D e tr ) = N j=1 t∈I,x∈Ω ∥(g θ (u j (x, t)) -G(f e (u j (x, t)), r e (u j (x, t))∥ 2 2 dxdt (3)with g θ an approximation function for the true dynamics. Our goal is twofold: first, to learn a function g θ capable of generalising to new trajectories from a set of known functions f e ∈ F tr , where e ∈ E tr ; and second, to efficiently adapt to new environments e ∈ E ad during which E tr ∩ E ad = ∅. The dynamics in the unseen environments are defined by functions f e ∈ F ad not encountered during training. From a meta-learning view, environments can be viewed as tasks we want to generalise to. In equation 2, the function G is typically unknown. We assume a simple form and consider two instances for decomposing the known and unknown parts of the PDE in our approximation model g θ . The first one is a simple additive decomposition:EQUATIONwhere g p ∈ G p encodes the incomplete physical knowledge and g c ∈ G c is the data-driven term complementing g p . It is assumed here that g p is incomplete but is an exact component of the complete PDE and g c (θ) is then modelling a residual term.Although simple, this decomposition can be shown to cover a large variety of situations (Yin et al., 2021) . We illustrate in figure 1 the difference between a complete and incomplete trajectory for Gray-Scott PDE a reaction-diffusion system where the missing part corresponds to the reaction terms.As a second instance, we considered the frequent case where a solver operating at a low resolution solves a simplified dynamics and has to be complemented to approximate the effects of unresolved small scales onto larger ones (Belbute-Peres et al., 2020; Kochkov et al., 2021; Um et al., 2021) . This framework is known as a closure model and is used for example in Large eddy simulation (LES), illustrated in figure 2 . By operating on larger scale dynamics than direct numerical simulation (DNS), LES offer a balanced compromise between computational cost and accuracy (Kochkov et al., 2021) .Since the closure model operates on the low resolution dynamics model, a better decomposition is: Training stage The objective is to learn the best environment-agnostic and environment-specific parameters. Ideally, we want the environment-agnostic parameters to capture most of the dynamics and only use the environment-specific parameters to capture information not shared across environments. We impose a locality constraint such that θ e remains small:g θ = (g c • g p )(θ) + g p (5)EQUATIONAdaptation stage All parameters except environment-specific parameters are fixed. We want fast and accurate adaptation to new environments: we employ a linear hyper-network during training and adaptation stage that operates in a low-dimensional space, facilitating low-cost adaptation:EQUATIONhere θ e = W a c e , with W a = (W 1 , ..., W dc ) ∈ R d θ ×dc , a shared weight matrix learnt during training across environments and c e , a context vector learnt during adaptation. Adaptation to new environments thus only requires learning a small dimensional code c e . Data-driven CoDA 1.08e-3 9.07e-3 3.37e-3 1.23e-3 1.54e-3 5.14e-1 3.92e-3 2.57e-2 CAVIA 1.21e-2 1.24e-2 6.36e-3 4.06e-3 1.07e-2 4.29e-1 6.49e-3 4.81e-2CoDA + Phys loss 3.36e-3 8.62e-3 1.99e-1 2.03 e-1 1.42 1.7 3.67e-2 2.28e-1 Ours 3.07e-5 7.17e-4 7.95e-6 5.23e-6 1.63e-4 9.46e-3 1.96e-4 5.86e-3Model-driven Incomplete PDE Solver 5.55e-1 6.10e-1 8.51 6.23e1 9.34 1.35e1 2.70e-2 1.50e-1Algorithm 2: Adaptation Pseudo-codeInput: {D e ad } e∈E ad with #D e ad = 1; fixed W a ∈ R d θ ×dc and θ a ∈ R d θ , c e = 0 ∈ R dc while no convergence do c e ← c e -η∇ c e ( e∈E ad L(c e , D e ad )) endOur approach is validated on representative ODEs/PDEs for the forecasting task. We assess the model performance on two key aspects.• In-domain generalisation: the model capability to predict trajectories defined by unseen IC on all training environments e ∈ E tr . • Outof-domain generalisation: the model ability to adapt to a new environment e ∈ E ad by predicting trajectories defined by unseen IC. We also report in-range time error (In-t) and out-ofrange time error (Out-t) to highlight the model's ability to predict dynamics within and outside the time horizon it has been trained on.Setup • Datasets We used two ODEs and two PDEs to assess the performance of our method: Lotka-Volterra (LV, Lotka (1925) ) and damped pendulum for the ODEs, 1D Burgers (Basdevant et al., 1986) and 2D Gray-Scott (GS, Pearson (1993) ) equations for the PDEs. For all the datasets, we assume that the system is only partially known (details on the known/ unknown terms are provided in Appendix D for lack of space). • Baselines We compare our approach with data-driven, hybrid and model-driven baselines. CoDA Yin et al. (2022) and CAVIA Zintgraf et al. (2019) are data-driven meta-learning frameworks for fast adaptation to new environments. We also extend CoDA into a hybrid version by adding to the original data-driven loss a physics-informed loss representing the known physics. This represents a soft constraint approach for integrating physics into a data-driven model, whereas our hybrid approach, which directly incorporates a PDE into the model, corresponds to a hard constraint approach. We also compare to the incomplete physical solver's performance when used alone. • Implementation We used MLPs for Pendulum and LV datasets and resolutiondependent ConvNets for Burgers and GS. We use a RK4 solver for the neural ODE and context vectors of dimension 2 for all datasets (more details are available in appendix E).We report out-domain and in-domain generalisation results in table 1 and 2 (in appendix A), respectively. For both in-domain and outdomain, we outperform the baselines across all datasets sometimes with 1 or 2 orders of magnitude, showcasing the effectiveness of our adaptation approach. CoDA enhanced with physical loss performs worse than the original CoDA both for in and out-domain. Using an incomplete physical loss is not a viable alternative for building hybrid systems, while directly incorporating PDE solvers as a hard constraint proves effective. Besides, as noted by several authors, optimising physics loss from collocation points is an ill-posed problem. The incomplete PDE solver low performance on all the datasets highlights the benefits of the hybrid + adaptation approach. • In-range and outrange temporal generalisation Results for both in-range and out-range time horizons are reported in tables 1 and 2 (in appendix A). As all models have been trained only on In-t horizon, error accumulates over time and leads to lower performance outside the training horizon. However, our framework consistently exhibits the best score across all datasets when extrapolating outside the training horizon. • Ablation in Table 3 (Appendix A) presents the model performance without the adaptation mechanism. Here, the model learns a single corrective term for all training environments and directly infers out-domain environments without adaptation. The performance is significantly lower than with adaptation, affirming the essential role of adaptation in modelling complex physics and underscoring the inadequacy of the ERM approach in this context. 
conclusion: 
 Our framework demonstrates the importance of adaptation for generalising to new environments and its superiority to the classical ERM setting. It also highlights the benefits of directly embedding PDE solvers as hard constraints in data-driven models compared to soft loss constraints. 
 
 Appendix: We report the results for in-domain generalisation in table 2. Across all datasets, we perform better than all baselines, often by 1 or 2 order of magnitude. Our approach achieves also better performance outside of the temporal horizon it has been trained on, despite a decrease in performance compared to the performance inside of training-horizon, due to the accumulation error of the auto-regressive method.In order to justify the importance of adaptation in our approach, we also evaluate the performance of our hybrid model but without any adaptation mechanism. This comes down to an ERM approach, where we evaluate our model on data coming different environments. We report the results of this approach in table 3 for both in-domain and out-domain trajectories. While an ERM approach can be used to generalise to the same dynamics it has been trained on, i.e in-domain results, it fails to predict trajectories coming from unseen environments (i.e. out-domain results) compared to adaptation methods. In some cases, an ERM approach achieves worse than an incomplete PDE solver alone, highlighting the importance of adaptation. Data-driven CoDA 3.39e-4 5.58e-3 7.91e-5 1.80e-5 1.61e-5 5.52e-2 3.81e-4 6.93e-3 CAVIA 4.86e-4 6.45e-3 5.10e-5 1.11e-5 2.54e-5 3.34e-2 6.50e-3 4.81e-2CoDA + Phys loss 3.98e-4 6.24e-3 7.08e-3 1.10e-3 3.84e-2 4.30e-1 4.48e-2 2.27e-1 Ours 4.39e-5 3.39e-4 6.90e-8 2.17e-8 5.07e-6 6.09e-4 1.77e-4 5.82e-3Model-driven Incomplete PDE Solver 5.35e-1 5. We review hybrid and generalisation methods for learning physical processes, notably in the context of dynamical systems.Purely data-driven approaches have emerged as a new tool for PDE solving (Raissi et al., 2019) and dynamics forecasting (de Bézenac et al., 2019) . While flexible and proficient in capturing intricate patterns (Li et al., 2021; Brandstetter et al., 2023; Serrano et al., 2023) , they tend to fall short when it comes to generalising to different operational conditions. This limitation arises from a lack of fundamental understanding of the systems they model. To unlock the full potential of deep learning in simulating physical processes, there is a growing need to incorporate physical insights into the deep learning framework.Data-driven with soft constraints Hybrid methodologies have emerged as a solution, combining machine learning techniques with physics losses (Wang et al., 2021; Li et al., 2023) . These hybrid approaches have gained traction for their ability to achieve robust generalisation. Within this framework, neural networks often play a crucial role, modelling specific aspects of conventional PDE solvers. However, when only incomplete knowledge is assumed, minimising an incomplete physic loss is not desirable.Correcting PDE discretization error Recent years have witnessed the development of deep learning models tailored for accurate simulation of turbulent flows (Kochkov et al., 2021) . These models leverage the training of neural networks with differentiable physics, addressing numerical errors inherent in PDE discretization. This highlights the capacity of neural networks to swiftly and effectively correct errors in under-resolved simulations. In our approach, we propose a more general formulation for learning incomplete physics or augmenting simplified PDEs.Incomplete physical models In alignment with the objectives of our work, some researchers have put forth frameworks aimed at augmenting incomplete physical dynamics with neural network models (Yin et al., 2021; Takeishi & Kalousis, 2021; Tathawadekar et al., 2023) . This trend reflects a broader effort within the scientific community to synergy the strengths of physical principles and deep learning for improved modelling and simulation of complex systems. However, none of those works tackle the problem of generalisation to new PDE parameters.In the context of dynamical systems, various methods have been explored to address the challenge of out-of-distribution generalisation (Yin et al., 2022) . Standard deep learning models excel at learning features aligned with the training data distribution but often struggle when faced with test data distributions significantly divergent from the training data.Meta-Learning One strategy to address this challenge involves the application of meta-learning approaches (Zintgraf et al., 2019; Wang et al., 2022) . These approaches seek to imbue models with general knowledge about the system by exposing them to a diverse set of M tasks, thereby enhancing adaptability to varying input distributions. Recent works have proposed a meta-learning formulation specifically for learning models capable of rapid adaptation to unseen tasks within the context of dynamical systems. We propose a similar approach, but extends to more complex dynamical systems where partial knowledge is assumed (Kirchmeyer et al., 2022; Wang et al., 2022; Park et al., 2023) .Incorporating PDE parameters Recent advancements in the pursuit of robust generalisation have introduced innovative techniques designed to bolster model performance in both long-term (Lippe et al., 2023) and out-of-distribution scenarios. Some of these techniques focus on the parameters of partial differential equations (PDEs) (Takamoto et al., 2023) , recognising them as pivotal factors influencing data distribution. Consequently, certain methods endeavour to explicitly encode the true parameter values into the latent space (Fotiadis et al., 2022) . However, most of these work only operate on ODEs or assume availability of larger amount of data.Generalisation with PINNs An alternative approach involves physics-informed neural networks (PINN) (Raissi et al., 2019) , which serve as PDE solvers. However, a limitation of PINNs is their inability to generalise to diverse scenarios. Therefore, ongoing research efforts have extended the capabilities of PINNs to facilitate generalisation across various system parameters, ICs and boundary conditions (Huang et al., 2022; Cho et al., 2023) . Approaches based on PINNs are data-free and thus assume complete knowledge of the PDE. In scenarios where only partial knowledge is assumed, soft constraints are ineffective compared to our proposed approach.In practice, f e is unavailable and we can only approximate it from discretised trajectories. As done in (Kirchmeyer et al., 2022) , we use a trajectory-based formulation of Eq. ( 3). We consider a set of trajectories discretised over a uniform temporal and spatial grid includes T ∆t ( s ∆s ) ds states, where d s u e j (s k , t l ) is the state value in the j th trajectory from environment e at the spatial coordinate s k and time t l ≜ l∆t. u e (t) ≜ [u(s 1 , t), . . . , u(s (S/∆s) ds , t)] T is the state vector in the j th trajectory from environment e over the spatial domain at time t and u e 0 is the corresponding IC.We present the equations and the data generation settings used for all dynamical systems considered in this work. In table 4 , we report all ODE and PDE parameters used to generate training and adaptation environments.The ODE represents the motion of a simple pendulum:EQUATIONwhere θ(t) is the angle, ω 0 the proper pulsation and α the damping coefficient. In our case where we suppose we only have incomplete knowledge of the phenomenon, we consider the term α dθ dt unknown and want to learn it using the data-driven component.We generate trajectories coming from different environments represented by specific parameter values for both α and ω 0 = ( 2π T0 ) 2 using Runge-Kutta 8 solver. For training, we generated 9 distinct environments, each composed of 16 trajectories on the time horizon [0, 10] with a time step ∆t = 0.5. For adaptation, we evaluate our method on 4 distinct environments defined by parameters unseen during training and trajectories time horizon is [0, 20]. Only one trajectory per environment is used to adapt the model to those new dynamics and we evaluate the model's performance on 32 trajectories per environments.The system describes the interaction between a prey-predator pair in an ecosystem, formalised into the following ODE:EQUATIONEQUATIONwhere x, y are respectively the quantity of the prey and the predator, α, β, δ, γ define how two species interact. Across all environments, we suppose α = γ = 0.5, with only β, δ varying across environments. For the physical model, the term γy is supposed unknown.We generate trajectories on a temporal grid ∆t = 0.5 using Runge-Kutta 45 solver.For training, we consider 9 different environments, each composed of 4 trajectories, each defined by specific ICs. For adaptation, we adapt our network on 4 new environments using only 1 trajectory per environment. We evaluate the adaptation's performance on 32 trajectories per environment. Trajectories for training have been generated on a time horizon [0, 10] and on [0, 20] for evaluation.Burgers' equation is a nonlinear equation which models fluid dynamics in 1D and features shock formation. We consider the following form of the Burgers equation:du dt + du dx = ν d 2 u dx 2 (13)where u is the velocity field and ν is the diffusion coefficient.Data generation For the DNS, we generate complex trajectories using a 5 th order central difference scheme using Runge-Kutta 45 solver with a time-step ∆t = 1e-5 and ∆x = 2π 16384 . Such trajectories are particularly costly to generate, therefore, we rather use LES. To obtain the ground truth LES trajectories, we apply a spatial filtering operator on the DNS trajectories. We also down-sample the temporal and spatial grid. Therefore, we obtain LES trajectories with a timestep ∆t = 1e-3 and ∆x = 1 256 . Applying such filtering operator to the Burgers equation leads to the following formulation:EQUATIONwhere R represents the unresolved subgrid scales, which are smaller than the resolved scales in the coarse grid. We try to learn these unresolved scales with the data-driven component. For training, we generate 4 environments with 4 trajectories per environment, each corresponding to a different viscosity coefficient. Trajectories have been generated on a temporal horizon [0, 0.05] for training and on [0, 0.1] for evaluation.The PDE describes reaction-diffusion system with complex spatiotemporal pattern through the following 2D PDE: EQUATION 


Decision: Authors propose a meta-learning approach to efficiently adapt to unseen context, applied to ODEs and PDEs. For the camera ready version, I recommend the following based on reviewer feedback: 1. citing similar prior work like CoDA 2. improving the novelty and clarity of the presentation and justify design choices. Authors should revise the framing to distinguish their contributions from prior meta-learning studies and provide more details on the model architecture and training methodology.

Reviews: review: title: title: Review of "LEARN TO ADAPT PARAMETRIC SOLVERS UNDER INCOMPLETE PHYSICS"
main_review: main_review: # Summary
The paper applies a meta-learning approach for efficiently adapting hybrid ODE and PDE models across multiple environments.

# Strengths
1. The problem studied by the paper, of generalizing hybrid models across environments, is interesting and impactful.
2. In general, the paper is easy to read and the figures, algorithms, and tables are well presented.
3. The range of experiments is good, and they appear to verify the effectiveness of the method.

# Weaknesses
1. As far as I can tell, the meta-learning strategy proposed here is identical to CoDA [1].
The primary contribution of this paper is therefore not the meta-learning strategy itself, but rather its application in a hybrid setting.
While applying CoDA to hybrid rather than purely data-driven models is a valid and interesting contribution, it nonetheless raises serious questions regarding the presentation of this paper.
In particular, **it would be easy to read the paper and think that the meta-learning strategy itself is a novel contribution**.
For example, the abstract states, 
> ...we introduce a meta-learning strategy that captures context-specific variations inherent in each system, enhancing the model’s adaptability to generalize to new PDE parameters and initial conditions,

    while the introduction says, 
    > To tackle this generalization problem, we propose a framework that can be viewed as a meta-learning approach for rapidly adapting the data-driven components of a hybrid mode [sic].

    In Section 2.3, "A Two Stage Framework", the paper's meta-learning algorithm is introduced *without any reference to CoDA*. 
    In fact, CoDA is referenced only in the Appendix, where the authors state,
    > We propose a similar approach, but extends to more complex dynamical systems where partial knowledge is assumed.

    It's possible that I've missed some novel aspect of the proposed meta-learning algorithm other than its application to hybrid systems; if so, it still needs to be made very clear how the proposed algorithm relates to prior work, especially CoDA.
    As it stands, I unfortunately have serious doubts about the originality and the presentation of this work.
1. Both CoDA and CAVIA are used as baselines but without reference to the relevant papers, requiring some detective work by the reader.

# Other Comments
1. In many places, such as the abstract and introduction, the paper is written as if it were about PDEs only, but the experiments consider both ODEs and PDEs. 
1. In the results in Table 1, it seems strange that "CoDA + Phys loss" performs strictly worse than CoDA, in many cases by multiple orders of magnitude.
If the physics loss were properly motivated, implemented, and trained, shouldn't it generalize at least as well as CoDA?

# Conclusion
Given my concerns about originality and presentation, outlined in the first weakness above, I cannot recommend this paper to be accepted in its current form.
In my opinion, the paper is presented as "a new meta-learning framework for hybrid models", while the reality is closer to "an existing meta-learning framework applied to hybrid models".
That being said, I still find the work relevant and interesting, and I look forward to seeing future versions.

# Citations

[1] Matthieu Kirchmeyer, Yuan Yin, Jeremie Dona, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing to new physical systems via context-informed dynamics model. *International Conference on Machine Learning*, 2022.
title: title: A meta-learning approach for quick adaptation to new contexts in PDE learning
main_review: main_review: This paper proposes a meta-learning approach for learning parametrised PDEs, wherein the PDE is hardly encoded, as opposed to soft-constraint approaches where it is used in the loss. The proposed methodology allows for quicker adaptation to unseen contexts. 

How do you compute $g_{p}$, i.e. the incomplete PDE part?

Also, references are missing for the CODA and CAVIA methods.
title: title: Review of "Learn to adapt parametric solvers under incomplete physics"
main_review: main_review: The manuscript titled "Learn to adapt parametric solvers under incomplete physics" outlines an approach to predict the time dynamics of a dynamical system by incorporating known components of the physical characteristics and modeling only the unknown components. The paper is well written and clearly outlines the challenges, the approach used and the rationale for all decisions made towards the final proposed solution. The key idea is to decompose the RHS of a PDE into known and unknown components with an assumption on the functional form of this split and only train a model to learn the unknown dynamics. These unknown dynamics are further split into environment-agnostic and environment-dependent portions and a meta learning approach is described to be able to compute the environment-dependent portion of the model parameters using a linear model and learned low dimensional context vectors. This low dimensional context vector enables cost efficient adaptation to new environments. The experiments are conducted are described well and the results and ablations show the appeal of the proposed approach. With the exception of a couple of small details, all the conclusions drawn from the experiment results are well substantiated. Overall, I would recommend accepting this paper. Some specific comments:
- Section 1: "incorporating physical constraints within a loss function" Should cite Raissi et. al. 2019 here since they were the first to do this.
- Section 2.2: "Since the closure model operates on the low resolution dynamics model, a better decomposition is:" Why is composition a better approach for closure models? Typically closure terms are added as an additional source term, so wouldn't the residual approach be appropriate here as well?
- Eq 7: Is there any specific reasoning or intuition to choosing the sum of \theta^e and \theta^a vs. some other architectural mechanism to produce g_c(\theta^e, \theta^a)?
- Section 3: Citations for CoDA and CAVIA are needed here.
- Section 3: "We also extend CoDA into a hybrid version by adding to the original data-driven loss a physics-informed loss similar to the ones in PINNs" Should clarify if the physics-informed loss is applied on the whole model based on incomplete physics or a residual type approach with one part having the incomplete physics loss and the other one being purely data driven. The latter would be more comparable to the approach here and allow direct comparison of soft vs hard constraints, but the discussion makes it sound like the former was used.