Med-Real2Sim: Non-Invasive Medical Digital Twins using Physics-Informed Self-Supervised Learning
Authors: Keying Kuang
Frances Dean
Jack B. Jedlicki
David Ouyang
Anthony Philippakis
David Sontag
Ahmed Alaa
Venue: NeurIPS 2024 poster (2024)
Keywords: Inverse Problems
Digital Twins
Personalized Medicine
Non-Invasive Imaging
Physics-Informed
Primary area: machine_learning_for_healthcare

Abstract: A digital twin is a virtual replica of a real-world physical phenomena that uses mathematical modeling to characterize and simulate its defining features. By constructing digital twins for disease processes, we can perform in-silico simulations that mimic patients' health conditions and counterfactual outcomes under hypothetical interventions in a virtual setting. This eliminates the need for invasive procedures or uncertain treatment decisions. In this paper, we propose a method to identify digital twin model parameters using only noninvasive patient health data. We approach the digital twin modeling as a composite inverse problem, and observe that its structure resembles pretraining and finetuning in self-supervised learning (SSL). Leveraging this, we introduce a physics-informed SSL algorithm that initially pretrains a neural network on the pretext task of learning a differentiable simulator of a physiological process. Subsequently, the model is trained to reconstruct physiological measurements from noninvasive modalities while being constrained by the physical equations learned in pretraining. We apply our method to identify digital twins of cardiac hemodynamics using noninvasive echocardiogram videos, and demonstrate its utility in unsupervised disease detection and in-silico clinical trials.

Med-Real2Sim, Non-Invasive Medical Digital Twins using Physics-Informen Self-Supervised Learning
Keying Kuang" UC Berkeley
Frances Dean" UC Berkeley & UCSF
Jack B. Jedlicki"
David Ouyang
University of Barcelona
Cedars Sinai
Anthony Philippakis' Google Ventures
David Sontag MIT
Ahmed Alaa UC Berkeley & UCSF
Abstract
A digital twin is a virtual replica of a real-world physical phenomena that uses mathematical modeling characterize and simulate its defining features. By con- structing digital twins for disease processes, we can perform in-silico simulations that mimic patients' health conditions and counterfactual outcomes under hypo- thetical interventions in a virtual setting. This eliminates the need for invasive procedures or uncertain treatment decisions. this paper, we propose method identify digital twin model parameters using only noninvasive patient health data. We approach the digital twin modeling composite inverse problem, and observe that its structure resembles pretraining and finetuning in self-supervised learning (SSL). Leveraging this, we introduce a plysicc-informee SSL algorithm that initially pretrains neural network on the pretext task of learning differentiable simulator a physiological process. Subsequently, the model is trained reconstruct physi- ological measurements from noninvasive modalities while being constrained by the physical equations learned in pretraining. We apply our method to identify digital twins of cardiac hemodynamics using noninvasive echocardiogram videos, and demonstrate its utility in unsupervised disease detection and in-silico clinical trials.
Introduction
With increasing health data availability, there is excitement about refining physics- based models of
human body systems to be patient specific, as personalized physical models can provide the basis for
in-silico experiments and timely diagnosis (27 551 25).. This personalized vision of healthcare has
given rise to virtual patients constructed with data tuned models, known as digital twins. Originally an engineering concept, digital twins have recently been realized as a resource healthcare (6. (). The concept combines data-driven approaches with mechanistic or simulation techniques, serving as a bidirectional map between real data and simulations. Medical digital twins have been applied and developed broadly in applications ranging from cellular mechanics (24) to the development of whole body and human digital twins (50) 76), with aims ranging from tailoring cardiovascular interventions (30) to agent-based trauma care management systems (14). Both academic and industry efforts are
prolific with digital twins having been developed for various medical problems ( 69 74 13 22 770).
In this paper, we propose framework for identifying patient-level digital twins using noninvasive medical images. We focus on scenarios where identifying a patient's digital twin from noninvasive data allows us to simulate other physiological parameters that typically require invasive measurement, such as through catheterization. Such an approach assumes that there a mapping from rich non- invasive imaging data to patient-specific mechanistic models of physiology: such mapping is not known to scientists humans but can learned from data. This hypothesis is supported by previous experimental work 2 64 43 However, the challenge lies in the absence of datasets containing both
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

noninvasive data and the corresponding parameters of the underlying physics-basee physiological model, hence the problem cannot be solved using standard supervised learning. To this end, we pro- pose a new class of physics-informa neural networks (PINNs) (60) to incorporate inductive biases informed by mechanistic models of patient physiology into neural network architectures.
Summary of contributions. We study obtaining la- tent parameters of physics-based model from image measurements directly an inverse problem. Learn- ing patient specific parameters for physics-based model from complete description of patient physi- cal states and a known forward model is an inverse problem. This task can be modeled using, for exam- ple, a PINN approach (71: 39 29). The process of obtaining patient physical states from non-invasive medical imaging is a second inverse problem with an unknown forward model. Obtaining patient specific latent parameters from non-invasive medical imaging directly combines these two inverse problems and could be modeled by learning the unknown process components. Large numbers of labeled data pairs of
Carduee hunctio
Dipial Tiies
Fill
WM-
PIC
Figure I: Digital twins for cardiac hemodynam- ics. Left: Illustration of the cardiovascular system. Right: Digital twin models of cardiac hemodynam- ies based on hydraulic or electric representations.
images and parameters for the full inverse task would need to be obtained from invasive procedures to use supervised learning. Instead, we propose training approach that structurally resembles self- supervised learning (SSL), and leverages labeled data pairs of images and non-invasively measurable patient physical states. We first train neural network using supervised learning with synthetic data to approximate the forward physics-basee -bsed model as a pre-text task. We then freeze this network to fine- tune the learning of a solution to the composite inverse problem from the labeled measurable data pairs. Our methods are then extendable to include exogenous intervention parameters for the development of in-silico experiments. We call this framework "Med-Real22 Real-Seml since learns virtual simulators of patient phyisology on the basis of real (noninvasive) data.
We apply our methodology in the setting of predicting patient specific cardiac pressure-volume loops from echocardiography. The relationship between left-ventricle pressure and volume is an important description of cardiac function (36). Pressure-volume catheter measurements are highly invasive, challenging their routine clinical use (77). Cardiac ejection fraction, the ratio between rettiventrc stroke volume and end-diastolic volume, while more routinely measured proxy for cardiac function, does not take into account cardiac pressure, preventing diagnosis of many severe cardiac conditions and providing motivation to develop digital twins for pressure-volume relations (80 45 (5).. Cardiovascular pressure and volume are commonly modeled using hydraulic analogy models, representing the hydraulic system by electronic circuits, characterized by lumped parameters corresponding to system attributes. Systems of differential equations describe the relationships between circuit parameters and volume and pressure states. model developed in (66) relates left-ventricular pressure and volume with and without the addition of a left-ventricllar assistance device (LVAD). We train neural network to learn to map between parameters and solution states of this model and use this learning to fine-tune a convolutional neural network that learns patient specific model parameters using echocardiography frames.
2 Method
2.1 Problem Setup
We consider a physics-based model M θ X' of a physiological process which maps set of parameters 0 E θ to a set of states x E X. The parameters 0 describe the physiological process in an individual patient; each patient i associated with a patient-specific parameter set 0, and physical states X4. We are interested in setup where the complete set of parameters and states can only be measured invasively, e.g.. through a catheterization procedure. We conceptualize the physics- based model M(0,) as the "Digital Twin" of patient i. That is, if we have oracle access to the parameters f; of patient i, then we can simulate their physiological states through the following forward process:
x = M(0).
(I)
The model M is assumed to be known and would typically be defined by a system of differential equations written fe(x) 0 with a physically interpretable parameter set O. One can use numerical
2

---

methods to solve fx(x) 0 for x and construct the solution M())) for each patient parameter set f; but a closed form solution define the map M may not exist even though the model known, such as in the case where So 0 has no closed form solution. The level of granularity in simulating the (true) physiological process depends on the fidelity of the physics-based model MM.
In addition to modeling the natural state of the physiological process, we also consider physics- models of clinical interventions (e.g., medical devices). To model an interventional process, we consider an augmented version of the physics-based model M θ - X, which maps the endogenous parameters 0 θ of the patient and exogenous intervention state x E X. The augmented model describes how the intervention u alters the physiological states of patient with parameters Ó- the causal inference literature, this class of models is known as structural causal models (SCMs) (57).
U to a physiological
To identify the model M(Ø) traditionally and simulate the states x, one would need to conduct a number of invasive procedures for each patient. Here, we assume that we have access to measurement y; for each patient which reflects some aspects of the underlying physiological process. We assume that y is acquired through non-invasive procedure that might conducted routinely clinical practice (e.g., an electrocardiogram) The relationship between the physiological state x and the non-invasive measurement y is given by
I((x).
(2)
Thus, by combining 1 and D we arrive at the following relation between the parameters of the physics-based model and the non-invasive measurements:
y
M (0) = F(0).
Unknown
Known
That is, the relation between the non-invasive measurements and the physical model parameters is composition of two functions; an unknown map IC that describes the relation between the physiological states and the observed measurements (e-g.. the relation between pixels in an imaging modality and underlying health states), and a known mathematical model M that captures the relation between the physiological states and unknown parameters 0. Our goal is to develop a learning algorithm to identify the digital twin M(0)) for patient given their non-invasive measurement y:.
Digital Twin Modeling as an Inverse Problem. Note that obtaining patient-speciff parameters 0, from physical states amounts to solving the following inverse problem:
0 = M 1MM-16 (x).
However, we do not directly observe the state x, but rather a measurement y obtained by passing x through the unknown forward model K. Thus, obtaining the physical parameters from y entails solving the composite inverse problem given by:
0 F= (y) = ---
Solving this problem entails two challenges. First, we do not know the forward model K. Second, we cannot directly learn the inverse map F=1 using supervised learning since this requires access to labeled dataset {y,, ). Obtaining such dataset would require conducting large number of invasive procedures, which we seek to avoid.
An alternative way to learn the inverse map F-1 is to break down the composite inverse problem in 5 into separate inverse problems, and only learn the inverse of the unknown map KC-1 using examples of the form ((y;,x;)) by fixing the map M. If we can construct mapping representative of M from any patient parameter set E to state x E X, then as we can write KC as follows:
K y -5-1 O - X
(6)
we can learn FF1 using training samples which only train K
As we will ultimately use supervised methods and gradient descent to learn Eq. (6). we need method for fixing M that allows for mapping any arbitrary parameter set 0 to corresponding state x. There are several methods in the literature for solving the forward dynamics of . that we might think of to construct this mapping. These include physics-informed neural networks (PINNs) (60), which train a network to approximate the solution x; to fo, (x,) = thereby solving M(0)) for a fixed patient parameter set 0,. Traditional PINNs learn the function x, by incorporating the differential
3

---

equation fe, = 0 as a coft-constraint in the network's 's training loss alongside a data loss. In the setting of a ordinary differential equations, Neural ODEs (TO) train a network learn the differential equation operator Jo, on the states x; to solve for x; M(0,) using differentiable ODE solvers. These methods can learn M(f.)) for a fixed patient parameter set 0g and can be compared to the use of numerical differential equation solvers.
These existent approaches for solving M, however, only give the states x for single patient at a time. To learn K-1 by training network through the map M, we need a method that approximates the map as otherwise, we would need to train infinitely many separate PINNs or Neural ODEs to cover the entirety of a plausible parameter range potentially traversed in supervised learning of (6)
Non-Invasive Data Acquisition. While we do not have direct access to the patient physiological state x; to train a model to learn 1 the noninvasive observation y, typically measures physiological quantities that can also be derived from the true state x,. We assume that x is a physiological variable that can be derived from both x and y, i.e.,
x = m(x) = g(y),
where m(.) is a known function and g()) is a labeling procedure, typically conducted manually by physician or automatically through built-in algorithm in the data acquisition device. This physiological variable represents a quantity of interest assessed by the noninvasive modality (e.g., breast density in mammography, blood pumping efficiency in ultrasound). These variables estab- lish a connection between the true state x and the observation y, as they can be both simulated from the physical model and measured based on y.
Given the setup above, our digital twin modeling problem can be formulated as follows. Given a dataset of noninvasive measurements for patients, D {y:) and physiological variable of in- terest & = m mx,) = g(y;), our goal is to train a model that can identify the underlying physics-base twin ((n++1) ) for new patient + 1 based solely on their corresponding noninvasive measurement
Yn+1- We seek to do this by fixing M and learning K. from the data D = {(y,, x;));-m-
2.2 Physies-Informet Self-Supervised Learning
Given the training dataset D {y,,, 8,) typical supervised learning task is to train model to predict the physiological variable &, on the basis of the noninvasive observation y, via standard empirical risk minimization (ERM):
f = arg min 1.
(8)
The motivation for training the model ʃ is usually to automate the collection of physiological variables from medical images in clinical workflow (68). These models follow a fully data-driven approach
and do not incorporate our knowledge of how the physiological processes being measured function
on the biological or physical levels. One could think of the supervised model f as a data-driven model of the function g(.) in 77.
Recall that our goal to recover the latent physical parameters 0 that underlie the physiologi-
cal processes that generated the observation y. If we had oracle access to dataset of the form D* {(y,, ( then a supervised solution to identifying the digital twin would be:
f arg min Σ ((fr))
The solution to (9) provides an approximate solution to the inverse problem in (5). As we lack access to D=, we can only learn the solution to (5) using the observed dataset D. To this end, we propose a two-step approach for learning the parameters 0 from y. leveraging the structural similarity between the composite inverse problem in (5) and the pretraniingffietuniig paradigm in self-supervised learning (SSL). An illustration of the two steps is provided in Figure 2(a).
Step 1: Physics-Informed Informed Pretext Task. In this step, we pretrain a neural network to imitate the forward dynamics of the physics-based model, i.e., x = M(0). We do so by first sampling ñ synthetic training examples from the physics-based forward dynamics as follows:
0,- ~ Uniform(O), x, = -((,) isish.
(10)
4

---

We denote this synthetic dataset by D= [(0,,,)]; Next, we use this dataset to train a feed forward neural network on the pretext task of predicting the patient physiological states from the physical parameters using ERM as follows:
Tm = arg min o
N/8 ((((), x)).
(11)
This training process distills the true physics- based model, MO), into the weights of a neural network δM. That is, the forward pass of the network will emulate the forward dynamics of the physiological process. i.e., x x 5M (0).
Step 2: Physics-Guided Finetuning. Given the observed dataset D {(y,, x;)]s, train another model to predict the physical parameters 0 from the observed measurements y using the loss function
ΦF arg min + (i (x))).
(12)
Here, the model pretrained synthetic data from the physical simulator, S.. is frozen and only the model oz is finetuned using real data on y and The neural network trained in Step 2, Cz. represents an approximate solution to the composite inverse problem in (3) i.e. Φ5(.) (.). For new patient n + 1, we discard the pretrained model S... and use the model कx to predict the patient's digital twin based their noninvasive measurement Ya+1 follows:
p = 25 6 Rat1 M(@m+1).
(13)
Interpretation of Physics-Informed SSL. Our proposed algorithm structurally resembles the SSL paradigm, where, akin to SSL, we decompose our model into "backbone" architecture and a task- specific "head" (47). However, there are fundamental conceptual differences between standard SSL approaches and ours. In standard SSL, the backbone is high- tigh-cppcity model pretrained on pretext task using unlabeled data to derive a general-purpose representation transferable to many downstream tasks, while the low-capacity head is finetuned for the specific task. In our physics-informed approach, SSL. serves to constrain rather than enhance flexibility. We first pretrain the low-capacity head to distill the laws of physics, and then finetune the high-capacity backbone model to learn mapping that aligns with the frozen head to produce the observed physiological variables while respecting the physical laws. In doing so, we force the backbone model to learn a mapping from the measurement y to the physical parameters 0.
The concept of distilling physics- based dynamic model into data-driven one draws from engi- neering methodologies that develop surrogate models to simplify computational complexity using simulated data (e.g.. (58)). This data-driven approach mirrors other supervised learning generative strategies for solving inverse problems or system identification with known dynamics in control theory (51 78). Importantly, physics-informed SSL scales efficiently with increasingly complex and higher fidelity physical models, as its computational requirements involve one offline process (Eq. (10) and 11 once the synthetic data is acquired.
3 Digital Twins for Cardiovascular Hemodynamics
We apply our physics-informed SSL method to perform patient-specific Pressure-Volume (PV) loop analysis in cardiology using echocardiography (cardiac ultrasound). Left ventricular (LV) PV loops illustrate the LV pressure against LV volume at multiple time points during a single cardiac cycle, providing comprehensive view of a patient's cardiac function and encoding various hemody- namic parameters like stroke volume, cardiac output, ejection fraction, myocardial contractility, etc., which can be used to diagnose cardiovascular discases (7). Traditionally, full characterization of PV loop requires an invasive PV loop cathetcrization procedure (35). Our objective is to utilize a noninvasive modality (i.c., echocardiography) to identify a digital twin for each patient, allowing us to simulate their individualized PV loops.
Physics-based model. We utilized a lumped- -arammetr circuit model of cardiac hemodynamics (i.e., blood flow), also known as the Windkessel model (79), as our underlying physical model ((). Lumped parameter models are based on the analogy between blood flow in arterial systems and the flow of electric current a circuit (Figure 1). The model comprises interconnected compartments that are equivalent to elements of an electric circuit. Resistances represent the resistance of blood flow
5

---

(a] Physics- informed self supervised learning
(b) Augmented cardiovascular and interventional model, M
Physics-informed Protest Task
Circuit model UVAD intervention
mm
Vaciion
X
Intornention asigement
LVAD
PM
Rs
W
RM
RA
Re
Step 2: Physics Guided Finetuning
m
O(t)
PM
22
24
233
Y
Dota driven
Prpsic--nformeed
Phytics Phyyics-hesed .M(9)
Figure 2: Illustration of Med-Real2Sim digital twins for cardiovascular hemodynamics. (a) Pictorial depic- tion of the two-step physics informed SSL algorithm proposed in Section 2.2 (b) Five state lumped lumped-parameter parameter electric circuit model of cardiac hemodynamics from (66). Here x(t) (t), (t)
22 (t), x²((), (t))), 24 (t),,, æs [t) (t), PA (t), (t) Q(t) describes the voltages T1, 22, Ta, 24 or pressures in the left-ventricle, left atrium, arteries, and aorta, respectively, and total flow æ5. The LVAD is modeled through an electric circuit connected to the digital twin via a switch. An LVAD intervention is applied the switch is closed.
within the blood vessels, accounting for pressure energy losses in the system; capacitors represent the amount of stored stressed blood; diodes describe mitral and aortic valves, and inductances describe the inertial effects due to blood acceleration (83). The blood flow within the system follows the standard Kirchoff's voltage and current laws. A patient's 's digital twin is represented by an electric circuit with a specific parameter instance 0, where corresponds to the components of the electric circuit. The patient's 's physiological state x corresponds to circuit currents and voltages (i.e., blood flow and pressures), which enables simulation of the patient's PV loop any local heart structure. In this study, we adopt the five-state electric circuit model from (66), illustrated in Figure 2(b).
The cardiac LV pressure PLv (t) and LV volume VLv are related by the elastance function, E(t) = (tv (tvtt(/Vv/tt)) - Va), which is modeled using a closed form in Eq. (5) in the Appendix. Evolution of x over time is governed by system of ODEs derived from current conservation laws (Appendix Eq. (20). The model is parameterized by a vector corresponding to bounds on the elastance, circuit resistance, capacitance, inductance and initial states (see Appendix A.2). Given the patient-specific parameters 6, for the ODE system and elastance function governing cardiac dynamics, we have unique solution for x,, i.e., M is injective (Appendix B
Modeling interventions. A sett-ventricular assistance device (LVAD) blood pump that helps improve cardiac function in severely ill patients. The effect of implanting an LVAD is modeled in with the addition of one state variable describing blood flow through the device and tuneable parameter. The impact of LVAD on blood flow can be modeled by attaching an exogenous electric circuit M()) form augmented model M()) (Figure 2(b)). Details on the mathematical model of LVVDD are in Appendix A
Noninvasive data. For each patient i, we have access to echocardiogram in the form of ultrasound video data y, and a physician labeled measurement of the LV ejection fraction X4. Echocardiography is widely used noninvasive modality for diagnosing cardiovascular diseases; it can directly provide volumes but not pressure measurements. Our goal is to use the echocardiography clip for a patient to predict their entire PV loop through a fully noninvasive process.
Experiments
4.1 Echocardiography Data
We test our physics-informed SSL -Med-Real2Simm approach using two echocardiography datasets: EchoNet and CAMUS. The CAMUS dataset (44) consists of 500 fully annotated cardiac ultrasound videos in 2-chamber view, each with LV volume labels for end systole and diastole (1 and Vi (l))). The videos were processed by spatial and temporal padding, with standardized 30-frame videos with a resolution of 256x 256 pixels. The EchoNet dataset (533 comprises 10,030 apical-4-chamber echocardiography videos from routine clinical care at Stanford University Hospital

---

also labeled with (tgs) and VLv These videos were cropped, masked, and resolution of 112x112 pixels using cubic interpolation. The CAMUS dataset was split into 450 training samples and 50 validation and testing samples. The EchoNet dataset was partitioned into 7,466 training, 1,288 validation, and 1, 1,276 testing samples.
4.2 Simulating individualized PV loops via digital twins
We train a 3D-CNN implemented using Pytorch to output subset of the model parameters 0z. The network consists of four convolutional layers, each followed by max pooling and concluded by convolutional layer with global average pooling and two fully connected layers. We choose seven of the model parameters listed in Table be learned in our model: mitral valve resistance RM. aortic valve resistance R.A, maximum elastance Emax, minimum elastance Emin. theoretical LV volume at zero pressure Va. start LV volume (O),(),, and heart cycle duration Tc. The remainder of the parameters are fixed to literature values (see Table 3). We enforce restrictions on the parameters by normalizing values in our activation layer, which scales parameters using sigmoid function to ensure parameters fall into realistic bounded ranges (as in Table 3). Parameters 6, are then passed to separately trained wxigh feed forward neural network ΦM to output the labeled values of & from which we construct training loss as the mean square difference between predicted and thysician-labeled VLV (tgs) and VLv(tED).
The network RM is pretrained by generating 3,840 synthetic data points linearly sampled from realistic parameter ranges mapping parameters 6g to VLv (TED) and VLv (tæs) using a numerical ODE solver (73). We use two fully connected layers trained to minimize mean square error between predicted and outputted (LEE))) and VLY (LES).
Once predicting 0,, we can numerically solve for the complete state vector x; for each patient. The first state in x; is the patient's estimated LV volume VLV. The elastance function (Eq. (T5) allows us to obtain LV pressure PLv (t) from volume. We plot the two functions PLV. VLV against each other over time obtain the PV loops. This process is deterministie, as the ODEs/clastance function are fixed by patient parameters 0; We also use VLV (tæD) and (tgg) to compute ejection fraction (EF) as EF (VLv (f&D) VLv (tes))/VLv (TED) We compare parameters and patient PV loops across EF, routine indicator giving information on cardiac function.
Our model has good correlation between labeled and simulated end-systolic and end-diastolic vol- umes. Our approach, Real/Simm achieves a mean absolute error (MAF) of 6.81% % for CAMUS and 5.40% for EchoNet in predicting EF (Table D. comparable to a baseline that uses a 3D-CNN and fully connected layers to directly approximate F-1 using supervised learning without passing the
MAE (EF)
Dataset
Supervised 3DCWV
Med Real2Sim
CAMUS EchoNet
6.58
6.81
5.62
5.40
Table I: Mcan Absolute Error (MAE) for true and predicted EF (in %) by supervised learning and simulated ejection fraction using Med- Med-Reel2SSim.
alysics-constrained layers d.-
Our model predicts PV loops that vary by labeled EF demonstrating that the model learns variability in patient physical state from our labels (Figure rightt). Predicted parameters also vary across patient EF in our model, and the relative variation in high- and low-EF patient groups were replicated across EchoNet and CAMUS (Figure 3). Notably, the relationship between the parameters and patient groups by EF concur with intuited patterns. Increases in Emax and Emin give an increased differential between pressure and volume, which is more plausible at higher EF, RM and RA are the circuit resistances associated to the mitral valve and aortic valve, respectively. In the mathematical model, increases in these resistances must either increase pressure or decrease blood flow by Ohm's 's law. Decreases in either resistance clinical is considered sign of increased hemodynamic burden, but its diagnostic value for indications including stenosis are controversial and a decrease in resistance is not necessarily predictive of reduce EF (2). In mitral regurgitation, valve resistance decreases, but EF is often increased in the beginning of the disease, which could contribute to the inverse pattern for predicted RM (52).
4.3 Unsupervised disease diagnosis
The PV loops simulated patient digital twins can serve as indicators for certain diseases that may not be directly labeled in the dataset. Variations in PV loops clinically enables the diagnosis of diverse cardiac abnormalities, which cannot be predicted using EF and LV volumes alone (55). We acquired physician labels for Mitral Stenosis (MS) for a subset of the patients in the EchoNet dataset. These
7

---

[a] Leamedd parameters of the physics based models EchoNet and CAMUS
(b) Simulated PV loops in patients diagnosed with mitral stenosis
Z:
Emaz
Ente
Ro
R2
V/
Awatge leea ow navidd
The reeeeruutee henits HG nor pataatt Aoced
Figure 3: (a) Learned parameters of the digital twin in high- and low-EF patient groups within the EchoNet an CAMUS datascts. (b) Comparison of average (simulated) PV loops in digital twins of non-Mitre Mitral Stenosis (MS) patients and MS patients. The plot illustrates differences in simulated hemodynamies in the two groups and agrees with theoretical PV loops for MS patients (48) Depicting of the theoretical PV loop for MS is courtesy
of/nttps::/ccppysioloogg www.//aart....... -isearrttddssaeee/00000000
labels were not included in the training data, and are not correlated with the EF measurement used to train our model (i.e., AUC of EF in predicting MS was 0.5). A total of 263 of 10,024 patients were identified as having MS. We randomly sampled 100 patients with MS and 100 patients without MS from the available dataset of 10,024 patients. We computed an average PV-loop for both the MS and non-MS groups and compared their respective averages analyze potential differences in Figure 3()). The model trend captures a distinctive patterns associated with MS: MS reduces LV pre-load and increases pulmonary venous pressures as illustrated in the right plot of Figure 3 b) 48). which aligns with the expected effect of MS on patients' PV loop patterns.
4.4 In-silico clinical trials
Using a patient's 's digital twin, we can simulate their counterfactual PV loops under hypothetical intervention in-silico to estimate its effect on the patient or to optimize the treatment parameters (c.g.. dosage) (56 72). Here, we demonstrate the feasibility of in-silico trial simulations under baseline and intervention conditions with the patient-specific digital twin M(#:) (Figures aa). In our setting, the lumped- parameter model of (66) is extended by the inclusion of one new state variable and five new parameters to model the addition of an LVAD. Tuning these new parameters and utilizing our model output allows us simulate the effect of LVAD on an individual patient in fully non-invasive manner.
The LVAD intervention has been shown to increase EF in vivo. We demonstrate the same result in-silico using the (66) model (Figure 4 The average EF for the CAMUS and EchoNet populations increase by 17.6% and 18.9% respectively, with the addition of an LVAD, consistent with reported findings (20 18)), with patients having lower pre-LVAD ejection fractions experiencing more sig- nificant increases. Figures c)) illustrates the distributional change of EF before and after LVAD implantation, indicating a significant right shift in the EF distribution following the procedure.
Furthermore, the influence of rotation speed on the PV loop dynamics is crucial factor in optimizing the therapeutic effects of LVADDs. The rotation speed determines the rate at which blood is drawn from the left ventricle and ejected into the systemic circulation, directly affecting the voluue relationship. Studies, such as those by Simaan ct al. (66), have explored the dynamic behavior of LVADs under varying conditions, including different initial rotation speeds. It is essential to note that the choice of rotation speed should be tailored to individual patient needs for optimal therapeutic outcomes. The impact of initial rotation speed on the PV loop, such changes in end- didsttoli and end-systolic volumes, should be carefully considered in the calibration process (Figure (b)).
4.5 Comparison of pnffire-i approaches
Our physics-informen pretext task is a computationally efficient generative approximation of the dynamics of the map M. We created second synthetic dataset of 1000 randomly sampled points to test the out of sample prediction of the physics-constrained layers ΦM and achieve an MAE of 2.30 on EF, The distribution of the loss is uniform across plausible parameter sets suggesting that our network is not subject to poor approximations in extreme cases (Appendix Figure D.. In contrast, other approaches to learn the forward dynamics of differential equation models include PINNs and
8

---

[x] Simulltted aftect of LXVO ome pictent
(b) Sirma abbed effect tuning pump spoed
Fressum fromidg
Walume On1)
Presuum frontig
MMW
Armid
Imsert of LVAD en the Echoltet CAMUS populations
m0
EchoWet
CAMUS
Figure 4: Counterfactual simulations of the LVAD intervention (left): (a).(b)(cc). PV loops for patients with normal, high, and EF (right).
Neural ODEs. As previously pointed out, these models are trained to learn the dynamics of a single patient, which incompatible with our goal of learning Eq. (6) with supervised learning.
We compare the ability our approach to learn the dynamics of the cardiac hemodynamics model for patients to PINN and Neural ODE methods and find it learns just well on fewer data points (Table 2}. Separate PINN and Neural ODE models are trained for each of
50 sets of patient parameters 0. We generated using numerical methods synthetic time series of 60 points for each of the five
states in the cardiac model: = [ From these synthetic data, we trained both PINN and Neural ODE approach predict x; for each of the patients computing average MAE on the 20 best results (Table 2) Recall that our method Φ.M M is trained with volume labels (one state) only. We found that only volume labels performed poorly in the Neural ODE settings (Appendix [2] PINNs were implemented in Tensorflow (6).. Neural ODEs were implemented in Pytorch using torchdiffeq (II).
Architecture
MAE
Training Time (per Epoch)
Cost per 9 Epoch
Model
Aug. EF
Pr
Specific Model (Avg. N=20)
Population Modil
Memory
PINN
I1f II (1x)(() (((1
7.67
0.16s
NA
O(N
O(N L)
(N=20)
Noural
fo.
(1x) - (x(1
3,39 (N
6.12s
NA
O(N)
O(N L)
PM
[1x((t) x011
O(L)
O(L)
(N=1000)
Table 2: Comparison of physies informed learning. We show the Mean Absolute Error (MAF) on ejection
fraction (EF). Here N is the number of patient parameter sets and is the number of layers in the network (6.M PINN) or the implicit layers i.c. function evaluations Neural ODE, Pt patient.
5
Conclusion
We present methodology, Med-Real2Sim, for non-invasive prediction of patient specific physics- based models directly from imaging that can form the basis for further development of medical digital twins. Our experiments showcase the ability of these methods to personalize a high fidelity physics-base model from video data. We demonstrate that information beyond labels learned in the setting of echocardiography for cardiac pressure-volumt loops. Our work may be extended to improve mathematical modeling for medical or non-medical applications. Latent parameters in our experimental setting are clinically informative, and our methodology gives non-invasive process for estimating their value. Further development of our methods could contribute new non-invasive direct ways of computing such parameters and provide the basis for in-silico digital twin studies.
Limitations We highlight that our model experiments are limited in their clinical validation. using only publicly available datasets with limited labels, we are not able compare measurements of heart resistance or fentrveent pressure, which would be necessary fine-tune such model for clinical use. While the ability to predict latent states and parameters is the core novelty of our research and

---

approach, acquiring further data to on these latent states could improve the model substantially. This a critical future direction for work in building digital twins with our methodology.
Acknowledgements
The authors would like to thank Ellen Roche, Caglar Ozturk, Yiling Fan, Luca Rosalia, and Yulun Wu for the helpful discussions.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 2146752.
References
[1] Abawi, D., Rinaldi, T., Faragli, A., Pieske, B., Morris, D. A., Kelle, S.. Tschöpe, C., Zito, C., and Alogna, A. The non-invasive assessment of myocardial work by pressure-strain analysis: 2022.
clinical applications. Heart Failure Reviews,
[2] Baldock, A., Rockne, R., Boone, A., Neal, M., Bridge, C., Guyman, L., Mrugala, M., Rockhill, J., Swanson, K., Trister, A., Hawkins- -aarud, A., and Corwin, D. From patient-speciff mathematical neuro-oncology to precision medicine. Frontiers in Oncology, 3, 2013. ISSN 2234-943X. doi: 10.3389/fonc.201 3.0000....
[3] Bastos, M. B., Burkhoff, D., Maly, J., Daemen, J., Den Uil, C. Ameloot, K., Lenzen, M., Mahfoud, F., Zijlstra, F,, Schreuder, J., et al. Invasive left ventriele pressure-volume analysis: overview and practical clinical implications. European heart journal, 41 (111)::1286-122 1286-1297, 2020.
[4] Björnsson, B., Borrebaeck, C., Elander, N., Gasslander, T., Gawel, D. R., Gustafsson, M., Jörnsten, R., Lee, E. J., Li, X., Lilja, S., et al. Digital twins to personalize medicine. Genome medicine, 12:1-4, 2020.
[5] Bora, A., Price, E., and Dimakis, A. G. AmbientGAN: Generative models from lossy measure- ments. In International Conference on Learning Representations, 2018.
[6] Bruynseels, K., Santoni de Sio, F., and den Hoven, J. Digital twins in health care: Ethical implications of an emerging engineering paradigm. Frontiers in Genetics, 9, 2018. ISSN doi: 10.3389/fgene. 201..00..........03....3
1664-8021.
[7] Burkhoff, D. Pressure-volume loops clinical research: a contemporary view, 2013.
[8] Burkhoff, D., Mirsky, and Suga, H. Assessment of systolic and diastolic ventricular properties via pressure-volume analysis: a guide for clinical, translational, and basic researchers. American Journal of Physiology- Heart and Circulatory Physiology, 289(2):H501 H512, 2005. doi: 10.1 10.1152/ajpheart.o 001 18001.8200 2005. PMID: 16014610.
[9] Canzoneri, M., De Luca, A., and Harttung, J. Digital twins: A general overview of the biopharma industry. Digital Twins: Applications to the Design and Optimization of Bioprocesses, pp. 167- 184, 2021.
[10] Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018
[11] Chen, R. T. Q. torchdiffeq. 2018. htt.///rge.//ggaa.b....ooo///aaa
[12] Choi, S., Boston, J., Thomas, D., and Antaki, J. F. Modeling and identification of axial flow blood pump. In Proceedings of the 1997 American Control Conference (Cat. No. 97CH36041), volume 6, pp. 3714-3715. IEEE, 1997.
[13] Corral-Acero, J., Margara, F., Marciniak, M., Rodero, C., Lonearie, F., Feng, Y., Gilbert, A., Fernandes, J. F., Bukhari, H. A., Wajdan, A., et al. The 'digital twin to enable the vision of precision cardiology. European heart journal, 41(48):45564466 2020.
[14] Croatti, A., Gabellini, M., Montagna, S., and Ricci, On the integration of agents and digital twins in healthcare. Journal of Medical Systems, 44:1-8, 2020.
10

---

[15] Cuomo, S., Di Cola, V. Giampaolo, F., Rozza, G., Raissi, M., and Piccialli, F. Scientific machine learning through physics--nformeeeee neural networks: Where we are and what's 's next. Journal of Scientific Computing, 92(3):88, 2022.
[16] Deguchi, S., Shibata, Y., and Asai, M. Unknown parameter estimation using physics-informet neural networks with noised observation data. Journal of Japan Society of Civil Engineers, Ser. A2 (Applied Mechanics (AM)), 77(2), 2021. doi: 10.2208/jseejam.771 77.22.....7.........
[17] del Ålamo, M. Deep learning for inverse problems with unknown operator. Electronic Journal of Statistics, 17(1):723 (11):723-7633 768, 2023. doi: 10.1214/23-ES22114
[18] Drazner, M. H. Left Ventricular Assist Devices in Advanced Heart Failure. Jama, 328(12): 1207-1209, 2022. ISSN 15383598. doi: 10. http:/jaaaaa22...33.222
[19] Dubois, A., Broadway, D., Stark, A., Tschudin, M., Healey, A., Huber, S., Tetienne, J.-P., Greplova, E., and Maletinsky, Untrained physically informed neural network for image reconstruction of magnetic field sources. Physical review applied, 18(6), 2022. ISSN 2331- 7019.
[20] Dunlay, S., Park, S., Chandrasekaran, K., Choi, J.-O., Pereira, N., Joyce, L., Daly, R., Stulak, J., and Kushwaha, S. Changes in Left Ventricular Ejection Fraction Following Implantation of Left Ventricular Assist Device as Destination Therapy. The Journal of Heart and Lung Transplantation, 32(4):S116, 2013. ISSN 10532498. doi: 10. he..0066...eallm2003.......
[21] Dupont, E., Doucet, A., and Teh, Y. W. Augmented neural odes, 2019.
[22] Erol, T., Mendi, A. F., and Dogan, D. The digital twin revolution in healtheare. In 2020 4th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT),
pp. 2020. doi: 10.1 NAL110/SSSS11366722200999555555.
[23] Ferreira, A., Chen, S., Galati, D. G., Simaan, M. A., and Antaki, J. F. A dynamical state space representation and performance analysis of feedback controlled rotary left ventricular assist device. In ASME International Mechanical Engineering Congress and Exposition, volume 42169, pp. 617-626, 2005.
[24] Filippo, M. D., Damiani, C., Vanoni, M., Maspero, D., Mauri, G., Alberghina, L., and Pescini, D. Single-cell Digital Twins for Cancer Preclinical Investigation, pp. 331-343. Springer US, New York, NY, 2020. ISBN 978-1-0716-01 59-44 doi: 10.10077978-1-0771-0000 1-07-6----77------
[25] Gottesman, O., Assadi, A., Mazwi, M., Shalit, U., Eytan, D., et al. ievs-inferring cardio-vasee vascular hidden states from physiological signals available at the bedside. Plos Computational Biology, 190)xe0118835-1000800
2023.
[26] Grandits. T., Pezzuto, S., Costabal, S., Perdikaris, P., Pock, T., Plank, G., and Krause, R. Learning atrial fiber orientations and conductivity tensors from intracardiac maps using physics-informe informed neural networks, In Ennis, D. B., Perotti, L. E., and Wang, V. (eds.), Func- tional Imaging and Modeling of the Heart, pp. 350-658, Cham, 2021. Springer International Publishing.
[27] Gray, R. A. and Pathmanathan, P. Patient-specific cardiovascular computational modeling: diversity of personalization and challenges. Journal of cardiovascular translational research, 11:80-88. 2018.
[28] Hand, P. and Joshi, B. Global guarantees for blind demodulation with generative priors. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Bue, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[29] Herrero Martin, C., Oved, A., Chowdhury, R. A., Ullmann, E., Peters, N. S., Bharath, A. A., and Varela, M. Ep-pinns: Cardiac electrophysiology characterisation using physics-informed neural networks. Frontiers in Cardiovascular Medicine, 2022. ISSN 2297-055X. doi: 10.3389/fcvm.200/ 2.211..

---

[30] Hirschvogel, M., Jagschies. L., Maier, A., Wildhirt, S. M., and Gee, M. W. An in silico twin for epicardial augmentation of the failing heart. International Journal for Numerical Methods in Biomedical Engineering, 35(10)=e3333, 2019. doi: tttps://doi.org/10. 10t0/0000.000//mm333333333 e3233 cnm.3233.
[31] Ienca, M., Ferretti, A., Hurst, S., Puhan, M., Lovis, C., and Vayena, Considerations for ethics review of big data health research: A scoping review. PloS one, 13(10):a0249937, 2018.
[32] Izgi, C.,, Ozdemir, N., Cevik, C., Ozveren, O., Bakal, R. B., Kaymaz, C., and Ozkan, M. Mitral valve resistance as determinant of resting and stress pulmonary artery pressure in patients with mitral stenosis: a dobutamine stress study, Journal of the American Society of Echocardiography; 20(10):1160-11 2007.
[33] Jagtap, A. D.. Kharazmi, E., and Karniadakis, G. E. Conservative Ptlysic--informe neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:113025 2020.
[34] Jin, K. H., McCann, M. T., Froustey, E., and Unser, M. Deep convolutional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9):409-4452, 2017. doi: 10.1 11110/TT2201..2713000
[35] Kass, D. A., Yamazaki, T., Burkhoff, D., Maughan, W. L., and Sagawa, K. Determination of left ventricular end-systolic pressure-volume relationships by the conductance (volume) catheter technique. Circulation, 73(5)5586-555, 1986.
[36] Kass, D. A., Md, M. M., Phd, W. G., Brinker Fsca, Jeffrey A., M., and Maughan, W. L.. Use of a conductance (volume) catheter and transient inferior vena caval occlusion for rapid determination of pressure-volume relationships in man. Catheterization and Cardiovascular Diagnosis, 15(3):192-20)) 1988. doi: https://doi.org/10.. 10..0///////...///c....... 1810150311
[37] Kharazmi, E., Zhang, Z., and Karniadakis, G. E. hp-vpinns: Variational physics-infonin neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering. 374:113547, 2021.
[38] Kind, T., Faes, T. J., Lankhaar, J..W.. Noorddord,,,,,f, A., and Verhaegen, M. Estimation of three-and four-element windkessel parameters using subspace model identification. IEEE Transactions on Biomedical Engineering, 57(7):1531-1538 2010.
[39] Kissas, G., Yang, Y., Hwuang, E., Witschey, W. R., Detre, J. A., and Perdikaris, P. Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive neural networks. Computer Methods in Applied
4d flow mri data using Mechanics and Engineering, 358: 12623, 2020. ISSN 0045-7825. doi: https://doi.org/10.10111 j.cma.2019.11663
[40] Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., and Mahoney, M. W. Characterizing possible failure modes in chysics-infonee neural networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang. P., and Vaughan, J. W. Advances Advances in Neural Information Processing
Systems, volume 34, pp. (6548-26560.. Curran Associates, Inc., 2021.
[41] Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D.,, and Matas, J. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[42] Lai, Z., Mylonas, C., Nagarajaiah, S., and Chatzi, E. Structural identification with physics- informed neural ordinary differential equations. Journal of Sound and Vibration, 508: 16196, 2021.
[43] Le, M., Delingette, H., Kalpathy- -Crmmer, J., Gerstner, E. R., Batchelor, T. Unkelbach, J., and Ayache, Mri based bayesian personalization of a tumor growth model. IEEE transactions on medical imaging, 35(10):2329-2) -2399, 2016. ISSN 0278-0062.
12

---

[44] Leclerc, S., Smistad, E., Pedrosa, J., Ostvik, A., Cervenansky, F., Espinosa, F., Espeland, T. Berg, E. A. R., Jodoin, P. M., Grenier, T., Lartizien, C., Dhooge, J., Lovstakken, L., and Bernard, O. Deep Learning for Segmentation Using Open Large-Scale Dataset in 2D Fchocardiography. IEEE transactions on medical imaging, 38(9):2198 2210, 2019, ISSN 1558254X. doi: 10. 109/TMLL2019.290055
[45] Marwick, T. H. Ejection fraction pros and cons. Journal the American College of Cardiology, 72(19):2360-2379 2018. doi: 10. 10.1016/j j0.c.1/j..ccc.2018.. 8.08.2162...
[46] Mascherbauer, J., Schima, H.,, Rosenhek, R., Czerny, M., Maurer, G., and Baumgartner, H. Value and limitations of aortic valve resistance with particular consideration low flow-low gradient aortic stenosis: in vitro study. European heart journal, 25(9):787-773, 2004.
[47] Misra, I. and Maaten. L. d. Self-supervised learning of gretext-invarian representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
6707-6717, 2020.
Motshabi-Chakane, P. Cardiovascular pressure-volume loops. Southern African Journal of [48]
Anaesthesia and Analgesia, pp. 1-5, 2023.
[49] Mousavi, A., Patel, A. B., and Baraniuk, R. G. A deep learning approach to structured signal recovery. In 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1336-1343, 2015. doi: 10. 1000///LETTO.220..........
[50] Okegbile, S. D., Cai, J., Niyato, D., and Yi, C. Human digital twin for personalized healthcare: Vision, architecture and future directions. IEEE Network, 37(2):262 269, 2023. doi: 10. 109/ 18.22000...
[51] Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R., Dimakis, A. G., and Willett, R. M. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1:39-56, 2020.
[52] O'Rourke, R. A. and Crawford, M. H. Mitral valve regurgitation. Current problems in cardiology, 9(2):1-52, 1984.
[53] Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz, C. Heidenreich, A., Harrington, R. A., Liang, D. H., Ashley, E. A., and Zou, J. Y. Video-based AI for beat-to-beat assessment of cardiac function. Nature, 580(7802):252-22) 256, 2020. ISSN 14764687. doi: 10. 10.1038/<41586- 1000:/44588000002220 020-2145-8-
[54] Pang, G., Lu, L.,, and Karniadakis, G. E. fpinns: Fractional informed neural networks. SIAM Journal on Scientific Computing, 41 A2203-26232222 2019. doi: 10.1 00.113//8M1120
[55] Pappalardo, F., Russo, G., Tshinanu, F. M., and Viceconti, M. In silico clinical trials: concepts and early adoptions. Briefings in bioinformatics, 2065): 2019.
[56] Pappalardo, F., Russo, G., Tshinanu, F. M., and Viceconti, M. In silico clinical trials: concepts and early adoptions. Briefings in bioinformatics, 1699)1169- 2019.
[57] Pearl, Causality. Cambridge university press, 2009.
[58] Pfrommer, J., Zimmerling, C., Liu, J., Kärger, L., Henning, F., and Beyerer, J. Optimisation of manufacturing process parameters using deep neural networks as surrogate models. Procedia
CIRP. 72:426-431, 2018. ISSN 2212-8271. 1. doi: https://doi.org/10..u pro.///proci..211....... 51st CIRP Conference on Manufacturing Systems.
[59] Potter, E. and Marwick, T. H. Assessment of left ventricular function by echocardiography. JACC: Cardiovascular Imaging, (art_Part__]]] 1):200)::224444 2018. doi: 10. 101g 017.
[60] Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707. 2019.
13

---

[61] Ramabathiran, A. A. and Ramachandran, P. Spinn: sparse, bassic and partially interpretable neural networks for pdes. Journal of Computational Physics, 445:1 2021.
[62] Sahli Costabal, F., Yang, Y., Perdikaris, P., Hurtado, D. E., and Kuhl, E. Physics-informed neural networks for cardiac activation mapping. Frontiers in Physics, 8:42, 2020.
[63] Sainte-Marie, J., Chapelle, D., Cimrman, R., and Sorine, M. Modeling and estimation of the cardiac electromechanical activity. Computers and Structures, 84:1743-1759, 2006.
[64] Scheufele, K., Subramanian, S., and Biros, G. Fully automatic calibration of tumor-growth models using a single mpmri scan. IEEE transactions on medical imaging, 40(1):193-2) 2021. ISSN 0278-0062.
[65] Seppelt, C., De Rosa, R., Mas-Peiro, S., Zeiher, A. M., and Vasa-Nicotera, M. Early hemodynamic changes after transcatheter aortic valve implantation in patients with severe aortic stenosis measured by invasive pressure volume loop analysis. Cardiovascular intervention and therapeutics, 37(1):191-2011 2022. ISSN 1868-4300.
[66] Simaan, M. A., Ferreira, A., Chen, S., Antaki, J. F., and Galati, D. G. A dynamical state space representation and performance analysis of feedback-comtrolleed rotary left ventricular assist device. IEEE Transactions Control Systems Technology, 17(1):15-22) 2008.
[67] Stergiopulos, N., Meister, J.-J., and Westerhof, N. Determinants of stroke volume and systolic and diastolic aortic pressure. American Journal of Physiology-Heene and Circulatory Physiology, 270(6):H2050-HH H2054, 1996.
[68] Stoyanov, D., Taylor, Z., Carneiro, G., Syyeda-Mahmood T., Martel, A., Maier-Hein, L., Tavares, J. M. R., Bradley, A., Papa, J. P.. Belagiannis, V., et al. Deep learning in medical image analysis and nultimodal learning for clinical decision support: 4th international workshop, dlmia 2018, and 8th international workshop, ml-cds 2018, held in conjunction with miccai 2018, granada, spain, september 20, 2018, proceedings, volume 11045. Springer, 2018
[69] Subramanian, K. Digital twin for drug discovery and development the virtual liver. Journal of the Indian Institute of Science, 100(4):653 (((4))653-66)))) 2020.
[70] Sun, T., He, X., and Li, Z. Digital twin in healthcare: Recent updates and challenges. DIGITAL HEALTH, 9:20552076221 149651, 2023. doi: 10. 10.117/200520762221444 149917777
[71] van Herten, R. L., Chiribiri, A., Breeuwer, M., Veta, M., and Scannell, C. M. Physics-informed neural networks for myocardial perfusion mri quantification. Medical Image Analysis, 78: 102399, 2022. ISSN 1361-8415. doi: https://doi.org) 00.//1...1111 202....mda......111..
[72] Viceconti, M., Henney, A., and Morley-Fletcher, E. In silico clinical trials: how computer simulation will transform the biomedical industry. International Journal of Clinical Trials, 3(2): 37-46, 2016.
[73] Virtanen, P., Gommers, R., Oliphant, E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E.,, Carey, C. J., Polat, I.. Feng, Y., Moore, E. W., VanderPlas, J., Laxalde, D., Perktold, J.. Cimrman, R., Henriksen, I., Quintero, A., Harris, C. R., Archibald, A. M., Ribeiro, A. H., Pedregosa, F,, van Mulbregt, P., and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020. doi: 10. 101.1////-1122-01--66666
[74] Voigt, I., Inojosa, H., Dillenseger, A., Haase, R., Akgün, K., and Ziemssen, T. Digital twins for multiple sclerosis. Frontiers in Immunology, 12, 2021. ISSN 1664-3224. doi: 10.3389/fimmu. 2021.669811
[75] von Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Pfrommer, J., Pick, A., Ramamurthy, R., Walczak, M., Garcke, J., Bauckhage, C., and Schuecker, J. Informed machine learning taxonomy and survey of integrating prior knowledge into learning systems. IEFE Transactions on Knowledge and Data Engineering, 35(1):614-66) 2023. doi: 10.1109/TKDDE.222..37777
14

---

[76] Wang, B., Zhou, H., Li. X., Yang, G., Zheng, P.. Song, C., Yuan, Y. Wuest, T., Yang, H., and Wang, Human digital twin in the context of industry 5.0. Robotics and Computer-Integraaee Integrated Manufacturing, 85:102626, 2024. ISSN 0736-5845. doi: http:///oii.or//1.. 101. 102626.
[77] Wei, A. E., Maslov. M. Y., Pezone, M. J., Edelman, E. R., and Lovich, M. A. Use of pressure- volume conductance catheters in real-time cardiovascular experimentation. Heart, Lung and Circulation, 23(11):1059-106) 2014. ISSN 1443-9506. doi: https://bbiorr//1..006....... hl.444 04.130.
[78] Werbos, P. An overview of neural networks for control. IEEE Control Systems Magazine, 11 (1):40-41, 1991. doi: 10. 10.11090 109/37.1033555
[79] Westerhof, N., Lankhaar, J.-W., and Westerhof, B. E. The arterial windkessel. Medical & biological engineering computing, 47(2):131-141, 2009.
[80] Westermann, D., Kasner, M., Steendijk, P., Spillmann, F., Riad, A., Weitmann, K., Hoffmann, W., Poller, W., Pauschinger, M., Schultheiss, H.-P., and Tschöpe, C. Role of left ventricular stiffiness in heart failure with normal ejection fraction. Circulation, 117(16):2051-2000 2008. doi: 10.1 CULLTCIICALAA 07..716886.
[81] Xu, K. and Darve, E. Physics constrained learning for data-driven inverse modeling from sparse observations. Journal of Computational Physics, 453:1 10938, 2022. ISSN 0021-9991. doi: http://doi..rr/10.100 2ttp://1...111.1....001.111111.0
[82] Yang, L., Meng, X., and Karniadakis, G. E. B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
[83] Yu, Y.-.. Minimally invasive estimation of cardiovaseular parameters. PhD thesis, University of Pittsburgh, 1998.
[84] Zhao, W. L., Gentine, P., Reichstein, M., Zhang, Y., Zhou, S., Wen, Y., Lin, C., Li, X., and Qiu, G. Y. Physics-constrainee machine learning of evapotranspiration. Geophysical Research Letters, (6424):14496-15500 2019. doi:
[85] Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., and Rosen, M. S. Image reconstruction by domain-transform manifold learning. Nature, 555(7697):487-492, 2018.
[86] Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 20/7 IEEE International Conference on Computer Vision (ICCV), pp. 2242-223 2251, 2017. doi: 10. 20.10099/
15

---

A Mathematical formulation of
hydraulic analogy model
The Simaan al. (66) model of the cardiovascular system is characterized by the values of the circuit elements, including the capacitances, resistances, and inductance, and by the initial conditions. The model is governed by 17 parameters: eight static parameters that account for the circuit's fixed properties, four parameters that capture the elastance function, and five values describing the initial conditions (see Appendix A.2). The elastance function relates the LV volume to the LV pressure
(t) E(t) (VLv(t) Va).
(14)
where Va is the patient specific theoretical volume LV at zero pressure. The elastance function is modeled in (66) as
La 0.7
1.9
E(L) (EMAX EMIN) 1.55
EMIN,
(15)
1.9
21.9
1
1.17
with t, = t/Tmax- where Tmaz = 0.2 + 0.15fe, and Te is the duration of a cardiac cycle. The evolution over time of the state of the circuit time / is described by 5-element vector
22, 33; 24, m5 PLvv, PLAY PA, PAos Qr]
(16)
where PLV,PLA. PA. PA, are pressures of the left- deft-ventricle, left atrium, arteries. and aorta, respec- tively, and Qr is total blood flow. We perform a change of variables from the (66) formulation of the model and write
x =
[a1, 22, 33, F4,, 25) (VLV - Va, PLA, PA, PAo. Qr)
(17)
for ease of computation.
In the model, t = 0 corresponds to end-diastole (ED), the end time of the filling phase. We assume the following constraints on the initial states
There no restriction on s/j, the initial difference between LV volume and Va. We write s (0) :=star_
For (0), the initial pressure in the left atrium, we know that the filling phase has finished, and the pressures in the left atrium and LV must be equal (63).
The initial arterial pressure PA (0), has to be equal to the aortic pressure as at ED there is no current leaving the LV nor the aorta, so the pressures must be equal.
The initial aortic pressure (A)(O). (0), has be larger than the LV pressure end diastole but by unspecified amount. We have a second free parameter, ppa that defines this pressure.
At ED there is not blood leaving the LV, so the initial total flow is (0) 0.
Thus, the initial state vector ED be written
x(t 0) |start v, start EMIN. start pao, start pao,
(18)
Under these assumptions, the model's 's degree freedom is constrained, and the number of indepen- dent parameters defining the system becomes 14 (8 for the static elements of the circuit, 4 capturing the elastance function, and for the initial conditions)
= R RA RC, Rs, L.g, CA. CR. Cs, FMAX EMIN; Te, Va, start_ start_pao).
(19)
The evolution over time of the state vector derived by application of Kirchoff's current law at five nodes, yielding the non-lincar system of equations
* A1x + Dip(x),
(20)
where A, a 5x5 time-independent matrix, and D is 5 time-independent matrix representing the activity of the diodes
0
0
0
RRCR
RyCR
0
1.
A
0
RgCs
RoCs
C5
;
DI
(21)
0
0
0
Cie
O
Ls
Ls
Lg
16

---

The vector p(x) is given by
max -(t)) 0)
p(x)= =
Rai
(22)
max 21 21-Ettt) 2o. 0
Ra
The first element of p is the current entering the LV, and the second element of p is the current leaving the LV. These currents depend on state of the valves, which depend on pressure differentials. Equation 20 combined with set of initial conditions x(),, provides the evolution over time of the volume the LV VV and the pressures in the system. The evolution of (t) and PLv (1) gives patient pressure-volume loops.
A.1 Addition of feedback-controlled LVAD
We simulate the addition of a left ventricular assist device (LVAD) to the cardiac system. One end of the device is connected between the left ventricle and the aorta, and the other end is connected between the left ventricle and the left atrium. Many different LVADs have been developed; we take here the case of rolary-controlled LVAD described in (12 666)
The presence of the LVAD induces the addition of new state variable, Zgg representing the blood flow through the device. In addition, the IVAD representation in the electric circuit is described by parameters Rg. R, P. a, L,, Lo. 3o. 31. 32. - describing compliances, static resistances and time-varying resistances and by the pump rotational speed, w(t). which is function of time. The pump speed can defined beforehand (e.g. a linear increasing function), or it can be updated in real-time based on the state of the system. A more detailed description of the parameters and equations of the LVAD can found elsewhere (12 (6).
By application of circuit analysis laws, new equations of the circuit describing the cardiovascular. LVAD system are given
A2(()y + Dap(y) + B())
(23)
for new state vector y, containing the first five variables as in 18 in addition to the blood flow through the LVAD, =t).). The new matrices of the system
0
0
0
0
0
0
0
0
0
R&CR
R
C
0
CR
A2(() =
0
0
RyCs
0
CyCss
0
0
0
C
CA
Ca
0
; DD =
0
O
0
-1
RG
0
Ls
L5
Ls
B(E)
0
0
0
R Ro Ru-Bo
L1
-L: Lo+31
Li Lo+
(24)
Rg is time-dependent resistance, only allowing current flow through the LVAD when the pressure in the left ventricle is below a threshold noted P. It is given by
R maxfa (#1 E(t) P),
(25)
The vector p(x) is as defined in and (t)) is given by
0
0
0
B(I)=
(26)
0
O
92
(())
+3
A.2 Parameters the sumped-paramedte hydraulic analogy model
B
Identifiability of the ODE inverse problem in the cardiac experiments
unique and valuable attribute the ordinary differential equation (ODE) our cardiac hemo- dynamics experiments is its identifiability. In this experiment setting, we can demonstrate that the model M well-defined and injective. To show this, we consider the inverse problem posed by finding parameters of the Simaan et al. (66) model, written M(0)
17

---

Parameter
the entire equation is Physiological Meaning
Reference Value
Allowed range
Parameters representing static element the circuit:
RM
Mitral valve resistance
0,0050 mmHg-s/m (66 83)
[0.005, 0.1] mmHg-s/ml)
RA
Aortic valve resistance
0.0010 s/ml
66
83
[0.0001, 0.25] mmHg.s/ml
Rc
Characteristic resistance
0.0398 mmHg
66
83
Fixed (0.0398 mmHg-s/ml)
Rs
Systemic vascular resistance,
1.0000 mmHg.s/ml
66
Fixed (1.0 mmHg-s/ml
related to the level of activity of the patient (23) Aortic compliance Systemic compliance
83
Ca
0.0800 ml/mmlig (66)
Fixed (0.08 ml/mmllg)
Cs
1.3300 ml/mmHg (66
Fixed (1.33 ml/mmHg)
CR
Left atrial compliance, repre-
4.4000 ml/mmHg 66 83
Fixed (4.4 ml/mmHg)
sents preload and pulmonary circulation (66) Inertance blood the aorta
Ls
0.0005 mmllg.s //ml 66
Fixed (0.0005 mmHlg.s/mm)
83
Parameters describing the elastance function of left ventricle:
EMAX
Maximum elastance, related
1.5-2.0 mmHg/ml (66), 2.31
[0.5, 3.5] mmHg/ml
to the contractility the heart (67
(6), 0.7-2.5 mmHg/ml (23)
EMIN
Minimum elastance
0.06 mmHg/ml 67
0.05
[0.02, 0.1] mmHg//l
mmHg/ml (66)
Va
Theoretical LV volume at
(23 12 (66) ml
[4.0, 25.0] ml
zero pressure
(67)
T.
Heart cycle duration (T: = 60/F H R)
0.8-1.0s (66)
[0.4. 1.7] beats/min)
(i.e.
35-150
Parameters describing the initial conditions:
start_v
Initial condition related (and not necessarily equal) to the ED LV volume
140 ml (66)
[0, 280.0) ml
start pao
Initial condition related (and not necessarily equal) to the ED aortic pressure
mmllg
Fixed (75 mmllg)
Additional parameters used in describing the addition the LVAD:
R.
Outlet resistance of cannulae
0.0677 mmllg.sfe (66)
Fixed (0.0677 mmllg.//m)
Ra
Inlet resistance of cannulae
0.0677 mmllg../he (66)
Fixed (0.0677 mmHlg./mll)
LVAD pressure parameter
-3.5//l (66)
Fixed (-3.5s/ml)
P
LVAD weight parameter
mmHg (66)
Fixed (1 mmHg)
L;
Inlet inertance of cannulae
0.0127 mmHg.s/ml (66)
Fixed (0.0127 mmHg.s/ml)
Lo
Outlet inertance of cannulae
0.0127 mmHg.s/ml (66)
Fixed (0.0127 mmHg.s/ml)
Po
LVAD dependent pressure pa-
-0.296 112
Fixed (-0.296)
rameter
81
LVAD dependent pressure pa-
-0.027
12)
Fixed (-0.027)
rameter
82
I.VAD dependent pressure pa-
9.9025 10 66
Fixed (9.9025 10-7)
rameter
Circuit pressure difference (inlet-outlet) defining L.VAD
H B0me + B1 day
Aow
pressure parameters
Table 3: Parameters the lumped-paren parameter hydraulic analogy model of the left-ventricle.
First, this function is well defined: given 0, which includes parameters describing the initial system state, there is unique ODE solution Using Picard-Lindelöe it is sufficient for A₁x + Dip(x) to be continuous discontinuous in é, the first condition is satisfied. As A,x D1p(x) is linear in each of the four cases defined by the pairwise possibilities for the rows of p(x) to either positive or zero, we can take the maximum needed Lipschitz constant amongst these four possibilities Lipschitz continuous.
and Lipschitz continuous in x. As x describes pressures and flows that cannot be
Conversely, given complete knowledge of x, we can find the twelve static model parameters in uniquely. We assume that complete knowledge of includes knowledge of the five state variables (VLv Va, PLA. PA, PAo, QT) over time in addition to the complete knowledge of PLv, VLV. The
18

---

difference between VLV and the first state variable gives Va. The ratio PLv/VLV - Va gives Et)).
The maximum and minimum values of this function give Emax and Emi.. respectively. The value of Te is the length of period of F(I). This gives the four parameters describing elastance.
Applying the Laplace transform 1(1)= Lo- c f(t)(tt to Equation 20 we have
(L))((s) x(0) (s) + DyL(p(x)).
(27)
The vector Lpp(x)) is constant. For any fixed s, L(x)(s) is also constant. Thus, each gives linear system of five equations in the eight unknown static parameters governing the Simaan et al. (66) model. Identifying these eight parameters uniquely simply requires choosing enough values of to solve each row as its own linear system. This shows that M is injective.
C Comparison of problem set up with existent literature
C.1 Ill-posed inverse problems
Generally, inverse problems are the task of determine inputs & E θ from measurements y E Y in the set up F(0) = y with forward operator F O Y. Inverse problems are typically ill-posed, meaning that the forward model F may be non-injective or that small perturbations of measurements y (i.e. noise) cause large change in the corresponding inputs 0. Often, variational approach is taken to mitigate these challenges, meaning the problem is reformulated into the task of minimizing over the possible solution set é the sum of a data fidelity term to represent fitting the model F in addition to a term incorporating solution criteria R at some level o:
min D(0, - RR()). BE0
(28)
For example, might be |IF(0) yl to encourage model fitting and R might be ||@]| to encourage input sparsity. This optimization task can been solved using analytical techniques to find 0 directly or using generative learning techniques to approximate F-1 to find
We also seek to solve an ill- ell-posed inverse problem as the forward model F is non-injective. There are different patient states that could look nearly identical on imaging. In our setting, as we further break down this ill- ill-posed task of inverting F as that of inverting F = K o M for components M θ - X and K. X + Y. we seek to solve an ill-posed inverse problem with partially known forward model. Other strategies for inverse problem with partially known forward model include adversarial methods such as CycleGAN for unpaired signals and measurements and AmbientGAN for training from measurements alone with parametric forward model. CycleGAN simultaneously learns the forward and inverse process by minimizes the expected value of the difference between ground truth inputs and outputs and with forward and inverse model composition outputs (86). AmbientGAN learns the inverse model by discriminating between the distribution the measurements and distribution of the forward model applied to the modeled inputs (5). For settings with unlabeled data, (19 pioneers Untrained Physically Informed Neural Networks (UPINNs), training a CNN to reconstruct images by reducing the training error between the original measurements and outputs passed through a known forward model.
These existent set ups differ from our task as our partially known model is really the composition of a known and unknown model. We take a generative inverse problem approach and develop a model for F-1, which we compare to other generative solutions in Table across the following two taxonomies: known versus unknown forward model A (row) and paired versus unpaired input-output data (column). This taxonomy mirrors that in (51).
C.2 Physics informed neural networks and related approaches
Efforts to incorporate knowledge into neural network models to improve training and accuracy include techniques to leverage physical models represented by differential equations (75). In big data settings, physical laws may be learned during modeling. In smaller data settings, incorporating physics into models can improve performance (75). This is especially pertinent to personalized medicine where data is inherently limited. Physics-informed neural networks (PINNs) typically have the goal of either finding solutions (forward) or system parameters (inverse) to systems of differential equations
19

---

Paired {Yr, 0.)3
Unpaired
Only {0,}
Only {y+)
Paired
No lim-
{Vr.
ited data
Known
DeconsttrucionSamm
Same training with paired y0,y)
UPINN
Traditional
techniques (34), De-
training with paired y)
PINNs for
Equation Discovery
noising
autoen-
(60)
coder
(49
Unknown Supervised
With data distribu-
F
learning 85
tions: (17).
CycleGAN (86)
Know
Blind de-
AmbientGAN
distribu-
modulation
(5
tion for
(28).
De-
F
blurGAN
(41)
Know M
Our method
Table 4: Comparison of problem set up and techniques for ill-posed inverse problems F(0) XCM((O)) = y. Abbreviations: UPINN= Untrained Physically Informed Neural Network,
GAN=Generative Adversarial Network.
60 15). PINN methodologies define training loss as the sum of a data loss and regularization term, which is the differential equation itself. There is extensive development and mixed success in PINN models for both goals (40).
The set up is the following. Given state vector E X dependent time variable and differential equation fx(x) = constraining the solutions x subject to a parametrization with initial or boundary data (0,, x). PINNs construct variational minimization task
min D x) + fo(x)
(29)
so that fx) acts as a regularizer. This framework can be used to solve both the forward task of solving J' for x and the inverse task of finding 0. The minimization is over the parameters of the neural network in the case of the forward task and over 0 E θ in the inverse task. We call the forward operator in this set up M, which the task solving the differential equation.
Several papers have used PINN approaches to study cardiac function, especially blood flow, including papers that use non-invasive medical imaging data. van Herten al. (71) learn parameters and solutions of a model for myocardial perfusion using measurements from MRIs. Kissas et al. (39) use a PINN approach to calibrate flow simulation model boundary conditions using data extracted from MRIs. Herrero Martin et al. (29) study cardiac electrophysiology with PINNs and demonstrate the ability of the method to learn latent variable. Sahli Costabal et al. (62) and Grandits et al. (26) use PINN approach to study cardiac activation. Our set-up differs in that we learn from images rather than through measurements obtained from images, and we predict informative latent parameters directly and seek a model that works for an entire patient population as opposed to single patient. Our methodology differs in that the physical model is not integrated into the loss function during training. Instead, we use supervised learning trained on data loss alone to generate forward model solving the ODE system and transfer this learning.
Drawbacks of the PINN approach include that the differential equation and boundary conditions need be exactly solved. Other approaches have solved fo hard constraint for these same aims in wwwssicscoossrainnee learning (PCL), which mitigates this challenge. A variety of variants on these traditional approaches continue to develop for both forward and inverse problems various specific problem set ups such as fPINNs (54), SPINNs 61), B-PINNs (82). VPINNs (37), and cPINNs (3) which while not directly relevant in our experiments may valuable future digital twin tasks. We compare related techniques to our with examples in Table C.2
20

---

Forward task (learn x(t))
Inverse task (learn ()
Traditional PINNs with f regularization term PCL: f as hard constraint
(39 (29 62
(26)
(3)
84
(1
PI-NODEs: Train neural network to learn the non- linear dynamics and a Neu-
42
ral ODE (10) to learn linear components
P-SSI: is used gener- ate synthetic data and use data loss for training gen- erative solution to inform larger inverse task
Our method
Our method could be in- verted for this task
Table 5: Comparison of techniques for embedding knowledge of physics models M(()) = x(t) neu- ral networks. Abbreviations: PINN-Physics-Infomee Neural Network, PCL=Physics Constrained Learning, SoODD===Pyysics-nfooome Neural Ordinary Differential Equations, P-SSL=Physics- Informed Self-Supervise Learning.
C.3
Neural ODEs
Chen et al. (10) proposed the use of differential equation solvers for solving neural networks upon noticing the similarity between ResNet gradient descent and Euler's method. Viewing neural networks as ODEs means forward propagation is the system solution. It also means that, in contrast, to ResNet architectures, the network does not have fixed number of layers. Neural ODEs are typically used to model time series data to replace ResNets. The set up is as follows.
Let ODESolver represent numerical ordinary differential equation (ODE) solver. A fully connected
neural network ff will represent the ODE the network learns to solve so that forward pass looks like
ODESolver(( f, to. To. T)
(30)
for initial condition x(4)) zo and prediction time series T. In this up. represent ODE
that we seek to approximate, analogous to our setting. Neural ODEs are trained by back propagating through the numerical ODE solver. Methods reduce the computational cost of this backpropagation include the adjoint state variable methods in (IO) and augmenting the system to higher dimensional space (2T).
Our fixed pre-training task for leveraging the physical model shares similarities with Neural ODEs in that train network to learn solve ODE. Our aim differs Neural ODE can learn the dynamics for a single patient. The ODE f will be specific to a single patient parameter set 6, but we seek to learn map for patient population.
D Electric circuit derivation of the (66) model
D.1 Deriving ODEs from circuit model
The Simaan et al. (66) model has five state variables which determined by solving system of ordinary differential equations (ODEs). We derive five ODEs from basic circuit current laws. We obtain equations for the derivative of each of the five state variables. Recall
x [ U2 33, Vi, 255
[Prv,, PLAY, PA, PAOy QT]
(31)
and that in our case, current is flow and voltage is pressure.
The following basic electric circuit facts are sufficient for the derivation.
For resistors: current voltage /resistance, i.e. / T= V/R.
For capacitors: current compliance derivative of voltage with respect time, ie.
I = dV de
21

---

For inductors: voltage = inductance derivativee of current wit t, i.e. V = di
Kirchhoff's is laws: current must be conserved at any node and voltage sums to zero around any loop.
current is the change in charge Q.
The first row of the ODE system is given by conserving the current at the red node in Figure 5 as
11 -
where 13 (C(1)=1)
(0) c(t) 31, I1 max(22 3ı, 0)/Ry RM the voltage flowing across the resistor would the difference between the voltage at the two capacitors (using Kirchhoff's law) and the diode prevents flow in the negative direction, and I2 max(x1 - I4, RA. Combining and rearranging, we get
5 (C) + max(222 x1, x1, ()/ max(21 x4, ()((())))
Rs
RM
RC
(())
Figure 5: Nodes for deriving circuit equations. The fifth row derived using total flow and inductance.
The second row is given as the following by conserving current at the yellow circled node in Figure
(xg - x2)/ Rg = Cna'y + max(T2 - x1, 0), Ras z/2 = (Iz x2) - max(x² - x1, x1, ()/ Cr).
The third row is given
Csrls = (x2 - 13)/ Rs + 45
- z2) / (RsCs) + as/Cs
by conserving current at the yellow node in Figure 5
The fourth row is given
CAR' -Is + max( - x4, 0)/RA
4' Is/CA + max(11 - I4, ()((RACA)
by conserving current at the green node in Figure
The fifth row is given using I
V
Lsd's = (I4 - 2z) + max(œ1 2zg 0)/
5/ = (Ir4 r3)/ Lss + max( 24, 0)/ R I.s).
This relation is written using the relation for inductors above the left hand of the first line is L di and the right hand side is voltage. In total, after performing change of variables of xı to dt oi == (t) these five equations give us Equation 20
D.2 Circuit Equivalence
We sought to find a reduced order equivalent circuit improve model performance but were unsuccessful in finding identifiable system. We began by ignoring diodes in our circuit to use linear equivalence, as then our circuit would have only linear components (capeitors, inductors, resistors), and we can use the follow result.
22

---

INO
RNO
C(I)
Figure 6: Norton equivalent circuit.
Theorem D.1 (Norton). Every linear circuit is equivalent to one with single current source and single resistor in parallel with a load of interest.
For our case this gives us circuit in Figure 6 as we are interested in the capacitor representing pressure change in the left ventricle.
To identify this circuit, we wrote the variables of this new circuit in terms of the state variables in the (66) model. To calculate the resistance of the equivalent circuits, we start by shorting all voltage sources (including capacitors) and opening all inductors. This gives us the circuit in Figure 7
Rs
RM
RA
Rc 0N
20V
C(t)
Figure 7: Calculating the resistance of Norton equivalent circuit.
As this is in fact the resistors Rs and RA in series and Ry in parallel, using current divider rules, the Norton resistance is
RNO
(32)
RM
Rs+ Ra
With the addition of diodes, since ours are ideal, when current flow the diode's forward direction, we replace the diode with short. In the reverse direction, we replace the diode with open, which would vary the calculated resistance to include and exclude resistances.
We would want to leverage the reduced circuit architecture to solve for (t). We know that the current through the load capacitor is Lep (CV (t))' using basic capacitor circuit relations. The Norton equivalent circuit gives node to solve using Kirchoff's ' law,
INNO = Leap + INO.
(33)
where Lavo is the current flowing through the Norton resistor. Thus, we write
INNO (t) RNO + (vv))
(34)
1
1
(tv
C(t) ( (-C'(t)Prv(t) InNO RNoo
(35)
giving two unknown state variables IRNO (t), (tv and one ODE, which is insufficient to determine the circuit.
E Training details
The EchoNet model, achieving a Mean Absolute Error (MAE) 5.40% was trained for hours over 13 epochs with batch size of 100 and learning rate of 0.001. In comparison, the CAMUS model,
23

---

which reached an MAE of 6.81% %, required hours of training over 110 epochs with batch size of 50 and learning rate of 0.005.
In our experiments, we conducted a comprehensive hyperparameter search to ensure optimal per- formance. We employed grid search approach systematically explore range of values for key hyperparameters. Specifically, we evaluated learning rates of 0,001, 0.005, and 0.0005, batch sizes of 50 and 100, and the number of epochs set to 300 and 500. We used the adam optimizer for training.
Models were trained on CPUs, each node featuring cores (12 physical cores with hyper-threading) and 128 GB of RAM.
F Additional figures from experiments
E1 Validation with Mitral Stenosis labels on EchoNet
PV Loop Comparison for MS and non-MS
160
120
100
60
40
IVV
Figure 8: Overlaid PV loops for non-MS patients and MS patients with red patients being labeled as having MS and black patients being without labels. Abbreviations: PV pressure-volume, MS=mitral stenosis, LVV = Left-ventriculae volume, LVP Left-ventricular pressure.
F22 Validation and comparison of physics-informed pretext task
volume End Diastole
LV volume at End Systole
250
Thue
150
30
50
100
150
200
250
Predicted
Figure 9: Testing the pre-training task on second synthetic dataset of 1000 randomly sampled points. The distribution of the loss is uniform across plausible parameter sets suggesting that our network is not subject to poor approximations in extreme cases.
24

---

[0] PINN patient example trained EC volumo labols after 100, OCO opocha
(b)) PINN socond patient exompic trained volume labela and 100,030 spocha
-30
345
25
4.1 4.5
4.50
417
6.35
4.50
4.1
4.55
Figure 10: Examples of aatient-specifie PINNs trained on synthetic data with 60 time points. Some PINNs learn the volume curve well. Others do not learn their patient curve after 100,000 epochs.
Naural ODEs trained 50 volume labels only for 30 spocns
Noural ODEs trained 60 labela for of five states
() Sampla patient
Volume Trajectoris
Volume Trajectories
All States
100
300
50
250
80-
200
70-
250
60.
30-
92
True x Wedictted
50-
1:00
50
0.2
0.6
0.0
1.0
1/2
10
50
(i) Secord sample patient
Volume Trajectories
All States
140
310
400
300
100
Tru
250
Predicted
200
300
100
80-
Tuo X
70-
Predicted
Da 10
20
Figure 11: Examples of patient-specific Neural ODEs trained on synthetic data with 60 time points for each of the fives states in the cardiac model or trained only on volume state only.
(a) Distributiin models used to caliullte the average in main taxt table
ejection fraction
PINN
(b) Distribution of ejection fraction 20 differrnnt patient Neural ODE models usad to calculate tha everage in main toxt table 2
10 12
Figure 12: Distribution of MAE on ejection fraction (EF) of the 20 best patient specific PINN and
Neural ODE models used create averages in Table 2.
25

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The abstract states that we propose a new non-invasive computational method for constructing digital twins and experiment with cardiac hemodynamics model. This is indeed what the paper does.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: have a limitation section pointing out the need for additional clinical validation of models built from these methods.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
26

---

Answer: [Yes]
Justification: There is no theoretical results in the main text. For the small number of theoretical results in the appendix. theorems used are clearly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
[Yes]
Justification: We provide a description the data processing and model architecture in the paper, and our code is publicly available GitHub.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
27

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
[Yes]
Justification:
We use publicly available data and our code is already publicly available on
GitHub.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so âNoå is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: Yes]
Justification: These are both detailed in the paper's appendix and able to be referenced in the released code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our experiments do not include the calculation of uncertainty in predictions. For experiments where we present an average across patients, we show the distribution of results in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include compute resources under training details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
We adhere to these guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss these implications in the introduction and conclusion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper does not release data nor use scraped data. While clinical use case is the subject of our experiments, the model is not intended to used any clinical setting discussed in limitations.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Data are cited. We use packages including ODE solvers which are cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification:
We have released our code along with its documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/coodemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
[NA]
Justification:
This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
[NA]
This paper uses publicly available datasets rather than directly interacting with any human subjects. Thus, this paper does not involve crowdsourcing or research with human subjects.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
31

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "Q9Yi7i7AP5", "reviewer": "Reviewer_LjNF", "summary": "The paper proposes a method to identify parameters for digital twin models of patients using non-invasive health data, eliminating the need for invasive procedures. This method focuses on scenarios like cardiac hemodynamics, where traditionally invasive measurements (e.g., through catheterization) can be predicted using non-invasive data (e.g., echocardiograms). The novelty of the method is to solve the associated inverse problem specifically for that patient, so that personalized predictions can be performed.\n\nThe proposed method uses a two-step SSL approach that structurally resembles pretraining and finetuning in SSL. First,  a neural network is pretrained on synthetic data to learn the forward dynamics of a physics-based model. Then the pretrained model is then used to train another network on actual non-invasive patient data to predict physical parameters.\nApplication to Cardiac Hemodynamics:\n\nThe paper illustrates how to apply the above method for cardiac hemodynamics using echocardiography. This allows the prediction of patient-specific pressure-volume (PV) loops using non-invasive echocardiogram videos.", "questions": "Have you experimented with this method beyond Cardiovascular Hemodynamics?\nYou implement a 3D-CNN - how did you arrive at such architecture? have you run ablations?", "limitations": "The authors mention the limited clinical validation. This is certainly one limitation to take in account. I would consider also the lack of generalization to different clinical procedures.", "rating": 3, "confidence": 3, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "The paper is overall well-written and it identifies a real problem with potential high impact: the design of personalised medical twins to avoid invasive procedures", "weaknesses": "The authors focus only on a very specific medical use-case, they don't try to generalize their method to more cases. In the introduction the authors illustrate this as a generic approach, therefore I was surprised to not find an attempt to support their vision with more application examples. Without the demonstration that this method can have a broader use I don't think this paper is suitable for presentation at this conference. Also, the authors don't run convincing ablations to demonstrate that their approach is sounding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the feedback on our paper! \n\nThe proposed method is indeed general and not limited to the cardiovascular system example. Any physical or biological system that can be described through ordinary differential equations can utilize our approach. The problem setup in Section 2.1 and the training/loss functions in Section 2.2 are designed to be very general, capturing any mapping from non-invasive observations to a physical system defined by differential equations. We chose to focus on cardiovascular hemodynamics as our example because our contribution is conceptual rather than empirical. By studying a specific disease area in depth, we were able to illustrate a wide range of use cases for digital twins, from diagnosis to simulation of counterfactual treatment outcomes. These use cases were previously not possible, and our goal was to demonstrate the potential rather than validate a new model for an existing use case at scale. We believe that our in-depth focus on a disease area enhances rather than undermines our contribution.\n\nTo give another example for a problem setup in oncology where the exact same framework can be applicable, consider the in-silico model proposed in [Baldock et al 2013], which describes the use of an ordinary differential equation for modeling proliferation and invasion of tumor cells. The two parameters of this model: dispersion D and proliferation p, have physiological meaning and impact on cancer prognosis [Baldock et al 2013]. They have been proposed to be measured using a mathematical formula from diffusion weighted MRI [Ellingson et al 2010 below]. Our approach could learn the dynamics of the ODE and build a non-invasive direct approach for these parameters to be extracted from images with a 3DCNN, provided data is available. By demonstrating a method for creating digital twins that can be generalized, our work could encourage more public sharing of relevant healthcare datasets, fostering the development of new digital twins across various medical fields. This is a crucial step towards broader applications and greater impact in medical research.\n\nIn response to your questions regarding the choice of architecture and ablation studies, we want to stress that our framework is not specific to this architecture. We found the 3D-CNN architecture to be the most empirically performant architecture in predicting ejection fraction using the PINN layers compared to other architectures we tried, such as LSTM and transformer models. This is in line with previous results that showed the effectiveness of 3D-CNNs in modeling echocardiography [Ouyang et al 2021]. Regarding ablation studies, we conducted an ablation by removing the physics-based model to assess its impact in EF predictions. **We are very open to suggestions from the reviewer for additional specific ablation studies** that could further validate our approach and their implications for our set up.\n\n_Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz, C.P., Heidenreich, P.A., Harrington, R.A., Liang, D.H., Ashley, E.A. and Zou, J.Y., 2020. Video-based AI for beat-to-beat assessment of cardiac function. Nature, 580(7802), pp.252-256._\n\n_Ellingson, B.M., LaViolette, P.S., Rand, S.D., Malkin, M.G., Connelly, J.M., Mueller, W.M., Prost, R.W. and Schmainda, K.M. (2011), Spatially quantifying microscopic tumor invasion and proliferation using a voxel-wise solution to a glioma growth model and serial diffusion MRI. Magn. Reson. Med., 65: 1131-1143. https://doi.org/10.1002/mrm.22688_", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oS7KvX7uOO", "reviewer": "Reviewer_32qb", "summary": "I have read this manuscript during ICML review. It looks the same so I copied my previous review.\n\nThe authors presented a method to infer the physical parameters θ of physiological process (heart pumping blood) from noninvasive observation y (the echo image). The mapping from y to θ cannot be directly learned due to the lack of paired data. Instead, they find that an intermediate variable x_bar (the EF) can be annotated by experts (x_bar=g(y)) which can also be calculated based on θ (x_bar=m(M(θ))). Thus the observable pair (y, x_bar) provides the supervision for learning θ = F_inv(y), through the relation x_bar=m(M(F_inv(y))) where m is rule-based and M is a solution to ODE. They first train a surrogate network to approximate M(θ) through synthetic simulation, followed by learning the parameter of F_inv on observations {(y, x_bar)}.", "questions": "1.Could the authors provide more solid validation of their inferred physical parameters?\n2.Could the authors provide the uncertainty/robustness/generalisation ability of the surrogate model?", "limitations": "My biggest concern is that the method lacks ground-truth validation, at least some samples.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written and easy to read with clear symbols.\n2. The idea of introducing a physical model helps better characterise the physiological system and provide a learning method.\n3. The surrogate model overrides the cost of solving ODE and make it differentiable during learning.\n4. The inference of physical parameters helps to perform virtual experiments.", "weaknesses": "1.The authors compared their methods in predicting EF from echocardiogram with supervised 3DCNN but did not outperform them in MAE. It should be noted that the EF is calculated on ED and ES segmentation and supervised segmentation network is expected to perform even better.\n2.Lack of validation. The authors performed validation by comparing EF derived from the physical parameters to the observed EF. But this does not guarantee the correctness of their physical parameters. In fact, arbitrary intermediate variables can be defined and could also lead to the comparable prediction of EF.\n3.Training the surrogate model of x=M(θ) need to generated synthetic samples while this is domain-dependant. Whether the learned mapping fits extreme or shifted situations is unknown, lacking of uncertainty analsysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review of our paper. We greatly appreciate your and other reviewers feedback from our ICML submission, and we have implemented changes to address these comments in our new submission. We are happy to share that we have **added additional validation and comparison of the physics based surrogate model in response to your feedback**. Specifically, we compared this approach to a baseline PINN approach and a Neural ODE approach, which demonstrate similar performance but fundamental differences in set up (Table 2). We also took into account your suggestion of further validating the surrogate model and its generalizability by adding predictions on an additional out-of-sample synthetic dataset in Section 4.5 and Appendix Figure 9. \n\nIn response to your specific concerns, we restate or add the following in a point-by-point response. \n\n**Weakness 1:** Thank you for your observation regarding the comparison with supervised 3DCNN and other segmentation models in predicting EF from echocardiogram data. While we acknowledge the potential for supervised segmentation models to yield superior results in terms of mean absolute error (MAE) for EF prediction, it's important to clarify that our study's primary objective differs from that of these supervised algorithms. In our research, the primary focus is utilizing the volume states that compose EF (volume at ES and ED) as a clinically significant label to facilitate the training of our model for latent parameters and PV loops. We incorporate physics as fixed layers for this purpose, a process which may marginally compromise the model's EF prediction accuracy. In Table 1, we showcase that the sacrifice in EF prediction accuracy is minimal and acceptable within the context of our objectives. We acknowledge that certain state-of-the-art Unet models designed for segmentation tasks may indeed achieve superior MAE in EF prediction. However, it's crucial to note that these models are specifically optimized for segmentation purposes, whereas our methodology integrates physics-informed learning to achieve broader objectives beyond segmentation alone.\n\n**Weakness 2:** Thank you for bringing up the limitation on the validation of our latent parameters. Evaluating solely comparing EF derived from physical parameters to observed EF for validation is clearly insufficient as you point out. Validating these individual parameters is of course challenging given that they are parameters of a physical model rather than exact patient measurements. In our particular experimental setting, our parameters certainly have physiological meaning, which we describe qualitatively, but they are not directly measurable. For example, Rm, mitral valve resistance, in our model is an electric circuit analogy parameter that is not equal to measures of mitral valve resistance in clinical practice. We sought to perform qualitative validation, i.e. describing qualitative relationships between Rm and actual mitral valve resistance and in using predicted PV loops to predict mitral valve stenosis disease labels. In future work, we would seek to find correlations between model parameters and physiological effects or conditions. For example, [Lamberti et al 2024 below] show correlation between pulmonary vascular compliance, a parameter similar to those in our model, and right heart decompensation with LVAD implantation. \n\n_Kimberly K. Lamberti et al, Dynamic load modulation predicts right heart tolerance of left ventricular cardiovascular assist in a porcine model of cardiogenic shock. (2024).DOI:10.1126/scitranslmed.adk4266_\n\n**Weakness 3:** The surrogate model/pretext task is definitely intended to be domain dependent and would need to be trained on synthetic data for any new in silico mathematical model. The data will always be synthetic for pretraining the physical dynamics, so it would not be dependent on the image data’s collection. In our case, the surrogate model is trained on a synthetic multidimensional grid filling a space of the form [a_1, b_1] x .. x [a_n, b_n], where [a_i, b_i] is the validity interval for the parameter θ_i, and θ=[θ_1,...,θ_n] and is chosen accordingly to the problem addressed. From this synthetic dataset, covering all possible states (and thus all possible patients), the surrogate model M(θ) is domain independent, and capable of predicting M(θ) even for extreme or shifted situations (since these are included in the synthetic dataset). We add an out-of-sample prediction on 1000 new points and show a uniform distribution of errors and comparison of this method to other physics informed approaches to demonstrate that it learns just as well and is better suited to our set up.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vJ72tbmO3L", "reviewer": "Reviewer_CAet", "summary": "This paper introduces a novel methodology for identifying patient-specific digital twins using noninvasive medical imaging, particularly focusing on cardiac hemodynamics. By leveraging a physics-informed self-supervised learning approach, the research addresses the challenge of modeling digital twins without invasive data collection. The process involves pretraining a neural network to simulate physiological processes and then fine-tuning it to reconstruct physiological measurements from noninvasive modalities, constrained by the learned physical equations. This framework allows for the simulation of patient-specific physiological parameters and the potential for conducting in-silico clinical trials and unsupervised disease detection.", "questions": "1. How do the results of the model change if the estimated parameters are dynamic rather than constant? Considering the medical context where a patient's condition is continually changing, assuming constant parameters seems unrealistic. How might this affect the reliability and accuracy of the model used in such scenarios?\n\n2. Can you explain why blood flow is often modeled using an electrical circuit analogy, and why it adheres to Kirchhoff's laws of voltage and current?", "limitations": "The proposed method requires a pretty strong assumption of the PDEs dynamics. I am a little bit worried about the training efficiency of this method. Can it be used as a real diagnostic tool? Additionally, I am intrigued by the potential of this method to be applicable across a broader range of medical disciplines.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a cutting-edge method combining physics-informed neural networks with self-supervised learning to tackle the inverse problem of estimating physiological parameters from noninvasive imaging data.\n\n2. By utilizing noninvasive data, the proposed method significantly reduces the need for invasive procedures, enhancing patient safety and comfort, and potentially broadening the applicability of digital twin technology in routine clinical practice.\n\n3. The methodology's ability to simulate detailed physiological conditions and interventions opens up vast possibilities for its application in personalized medicine, including unsupervised disease detection and in-silico clinical trials, which can significantly accelerate the development of therapeutic strategies.", "weaknesses": "1. Insufficient performance comparison. The paper only compares PINN and Neural ODE methods. There are many variations of PINNs methods which outperform the original one. The paper should choose more solid baselines.\n\n2. No ablation study. The paper proposed a two-stage training strategy, but it didn't show why it is necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for your very thoughtful comments! Below, we provide a detailed response to the weaknesses and questions. \n\n**Weaknesses:**\n\nPlease see below a point-by-point response to your concerns.\n\n* The limited baseline comparisons stem from the novelty of our problem setup. While there are many variants of Physics-Informed Neural Networks (PINNs) in the literature, not all are suitable for our specific problem. Our setup is unique because it involves a partially known model that combines both known and unknown components. As explained in lines 721-725, this setup can naturally be approached using a generative inverse problem approach to model $\\mathcal{F}^{-1}$. We compare this model to other generative and PINN solutions in Table 4, using two taxonomies: the known versus unknown forward model (rows) and paired versus unpaired input-output data (columns). Table 4 highlights that the various PINN variants make different assumptions about the problem setup, making them unsuitable as baselines for our case. Therefore, we chose to compare our approach only with baseline PINN and Neural ODE methods. Our goal was to demonstrate that our surrogate model achieves a different goal but still has a comparable understanding of the underlying physics as these standard approaches, rather than showing superior performance in learning physics. We are open to implementing additional baselines if the reviewer suggests specific PINN variants that are applicable to our setup.\n* We did conduct an ablation by removing the physics-based model to evaluate its impact on ejection fraction (EF) predictions (Table 1). Additionally, we explored other feature extraction architectures, such as LSTM and transformer models, but these did not achieve the same level of loss minimization as our 3D-CNN approach. For the two-stage training strategy, we also experimented with alternative differentiable ODE solvers. However, these solvers were not compatible with our architecture and resulted in similar parameters across different patients or introduced additional numerical error. Consequently, we opted for our two-stage approach, which includes a physics-informed pretext task and physics-guided fine-tuning with 3D-CNN. We are very open to suggestions from the reviewer for additional ablation studies that could further validate our approach and their implications for our set up. \n\n**Responses to questions:**\n\n**Question 1:** First, we want to clarify the difference between the patient's cardiac states and cardiovascular parameters. Our model is dynamic: it analyzes an ultrasound video to extract a cardiac state that varies over time, characterized by changes in pressure and volume throughout the video, which lasts for several minutes. The underlying parameters, however, are fixed because they depend on the heart's anatomy, deformities, and tissue properties, which change over days, months or years, not within the short duration of a single echocardiogram acquisition. While modeling the long-term evolution of cardiac health is a fascinating problem, our model can still address this setup by applying our two-stage procedure to new echocardiograms acquired over different years. \n\n**Question 2:** The analogy between cardiac blood flow and electric circuits is a classical result in hydrology and fluid mechanics. In this analogy, blood flow is analogous to electrical current as it is driven by pressure differences and encounters resistance like electricity is driven by voltage and material resistance. Valves act as diodes, capacitors emulate the elastance behavior of heart chambers, and heart tissues induce resistance analogous to that in electric circuits. This analogy is often referred to as the “drain-pipe theory” and is widely used in fluid mechanics beyond cardiovascular modeling. The class of models studied in our paper, called Windkessel models, with varying complexity have been used to model cardiac dynamics for many years, though other classes of models do exist. The model we take to be fixed for our paper was validated in the Simaan et al paper (which we specifically used for its incorporation of an LVAD). The authors synthetically perturbed the model to demonstrate the behavior was as clinically expected and compared to real human measurements to demonstrate accuracy.\n\nKirchhoff's laws are adhered to in the model simply because they are adhered to in any electric circuit. In this way, Kirchhoff's Voltage Law parallels the conservation of energy in the circulatory system, where the sum of pressure drops around a closed loop equals the pressure driving the flow. Kirchhoff's Current Law reflects the conservation of mass in blood flow, ensuring that the total blood flow entering a junction equals the total flow exiting.\n\n**Finally, we want to address your overall comment** and clarify that we do not make strong assumptions about PDE dynamics. Instead, we use a low-fidelity model that exactly characterizes cardiac hemodynamics in the heart chambers relevant to cardiovascular diseases, while ignoring the details of blood flow in other arterial peripherals. We strongly believe that our approach has broad applications for individualized treatment of end-stage heart failure. This includes not only the use of LVAD, as demonstrated in Section 4.4, but also any other medical device with a corresponding in silico model. Additionally, our method can be used to diagnose diseases not typically identified using echocardiograms alone, such as mitral stenosis.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nl8lyDx1ON", "reviewer": "Reviewer_8RJZ", "summary": "This paper proposes a novel method for creating patient-specific digital twins using non-invasive patient health data. The authors introduce a physics-informed self-supervised learning (SSL) algorithm that pretrains a neural network on learning a differentiable simulator of the cardiac process. Then, another model is trained to reconstruct physiological measurements from non-invasive data while being constrained by physical equations learned during pretraining. The method is applied to identify digital twins of cardiac hemodynamics using echocardiogram videos, showing good results in unsupervised disease detection and in-silico clinical trials.", "questions": "* Lines 251-255, Table 3: Which criteria was used to select the learnable and the fixed parameters of the model? Are these selected by using some kind of sensitivity analysis based on the state-space matrices in Eqs. 21 and 24?\n* Line 708: Appendix C1 already addresses the ill-possedness of the inverse problem. That means that the trained model could potentially assign the same digital twin to two different patients due to the similarity of their echocardiograms. Have the authors found any difficulty in this regard? \n* Have the authors considered using an easier and more accessible nonintrusive techniques such as electrocardiograms? Would the use of several modalities of non-intrusive data for the same patient increase the performance of the model?\n\nMinor comments:\n* Line 227: \"tune-able\" might refer to \"tunable\".\n* Line 252: Incorrect reference, Table A.2. might refer to Table 3.\n* Line 814: Incorrect reference, Figure D.2. might refer to Figure 7.\n* Figure 7: The figure model seems to be incomplete/offset.\n* Equation 35: A parenthesis is missing after $R_{NO}$.\n\nFinal comment: The paper is well structured, the results are promising and the methodology has moderate impact on the field. However, it's still unclear if this model could handle the variability and complexity of human physiology by only non-invasive measurements. Based on the comments above, I reccomend a weak accept.", "limitations": "Limitations were addressed appropriately.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The method uses non-invasive data, avoiding time-consuming and complicated patient interventions.\n* The model accuracy is enhanced by including a physics-based model during training.\n* The authors demonstrate the method's utility in modeling complex physiological processes like cardiac pressure-volume loops with open-sourced datasets.", "weaknesses": "* Simplifications/assumptions in the Windkessel and LVAD models might not fully capture the complexity of the heart dynamics.\n* The results might be sensitive to low quality non-intrusive data, which can affect the global accuracy of the method.\n* It is not clear the demographic diversity of the echocardiography dataset. Thus, the model might not generalize well across all the segments of the population.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very thoughtful comments and feedback! \n\n**Weaknesses:**\n\nThank you for highlighting these issues. Please see below a point-by-point response to your concerns.\n\n* While the Windkessel model does not fully capture the complexity of cardiac dynamics, this simplicity is actually a strength of our approach. The heart is an incredibly complex organ system, with millions of parameters describing its electrophysiology, hemodynamics, and biomechanics. However, it is not always necessary to simulate the entire cardiovascular system to make specific diagnostic or treatment decisions. Our model focuses on predicting pressure-volume (PV) loops relevant to diagnostic and treatment procedures for end-stage heart disease. In this context, the Windkessel model is the most parsimonious in-silico simulator for the relevant physiological processes. Additionally, we want to stress that our main contribution is a general two-stage physics-informed self-supervised learning approach and not a specific in-silico model—this approach is general enough to incorporate any cardiovascular system simulator, including more complex lumped circuit models of the arterial system.\n* Please note that non-invasive data is not necessarily of low quality; it simply means the data was acquired without invasive procedures, such as catheterization. Echocardiograms, which we use in this paper, are a widely-used, gold-standard non-invasive method for diagnosing cardiac function. Our paper proposes a new method that extracts additional valuable information from echocardiograms, which previously required invasive procedures and were impractical in most primary care settings. This method enables new use cases and does not compromise the accuracy of any existing procedures.\n* This dataset includes a reasonably diverse demographic, comprising 68% White, 14% Black, 7% Asian patients, and individuals from other ethnicities. Additionally, 55% of the patients are male. We will provide a detailed demographic performance breakdown in the Appendix of the final paper.\n\n**Responses to questions:**\n\n**Question 1:** We selected learnable versus fixed parameters based on the intended use case of predicting cardiac left ventricular pressure-volume (PV) loops in combination with a sensitivity analysis to understand how they affect the model. Ultimately, parameters directly related to the circuit analog of blood flow in the left ventricle were allowed to be variable, while others were fixed to isolate the prediction of these dynamics. We first defined intervals of validity for each of the 14 independent parameters, based on how variations of each parameter affects the PV loops (realistic shape and magnitude of pressures of volumes), while ensuring that the volume ranges and EF ranges of the two echo datasets were covered. These were centered around reference values from the literature. For the 14 independent parameters, we did perform a sensitivity analysis based on the state-space matrices in Eqs. 21. We selected the final 7 learnable parameters, based on how changes in these parameters affect the volume space, their physical meanings, their resulting PV loop shapes, and their ability to encompass the volume ranges of the two echo datasets. We **include a table in the global rebuttal PDF that summarizes the sensitivity analysis results and modeling choices**.\n\n**Question 2:** It is definitely true that, in general, patients with similar echocardiograms may be assigned identical digital twins and it is also true that this may not capture sufficient differences between patients. This is a general issue encountered in all inverse modeling problems, and is not peculiar to our setup. In our setting, we use a lumped parameter model with a small set of parameters that we have reasons to believe will be able to be distinguished between patients because they have a visual representation in an echocardiogram. This means that there are correlations between visual features and values of the in-silico model parameters that allow us to uncover meaningful differences in the digital twins that can distinguish, for example, mitral stenosis (Fig. 3(b)). We believe that studying the identifiability of specific models, developing general conditions for identifiability of physics-based models and developing new learning procedures that improve the chances of model identification are all very interesting directions for theoretical research that complements our applied method. One potential direction to improve identification would be incorporating patient level meta-data or other modalities as is done in other digital twin settings to enhance the model’s ability to distinguish between patients to make a measurement model more nearly injective. \n\n**Question 3:** It would be really interesting to study the use of electrocardiograms in this setting and even more interesting to combine these data assets. Echocardiograms were the natural choice for our model because they are a standard tool for encoding information about left ventricular pressure in addition to left ventricular volume, which they are routinely used to assess [Popescu et al 2022]. However, we recognize that incorporating additional non-invasive modalities such as electrocardiograms could potentially enhance the model’s performance. We are curious to explore the possibility in future studies.\n\nThank you for catching the references and minor typos, we will happily update those. Figure 7 is intended to show shorting the circuit to calculate an equivalent circuit.\n\n_Bogdan A Popescu, Carmen C Beladan, Sherif F Nagueh, Otto A Smiseth, How to assess left ventricular filling pressures by echocardiography in clinical practice, European Heart Journal - Cardiovascular Imaging, Volume 23, Issue 9, September 2022, Pages 1127–1129, https://doi.org/10.1093/ehjci/jeac123_", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their valuable feedback. We would like to take this opportunity to summarize the key contributions of our work, address common concerns across the reviews, and clarify some aspects of our methodology that may have not been fully appreciated.

**Summary of Contributions**
Our paper introduces methodology for tuning patient-specific parameters of medical digital twins with non-invasive data. We do so by creating a framework for solving a composite inverse problem to reconstruct physiological states from indirect measurement non-invasively. We build a physics-informed self-supervised learning (SSL) algorithm that first pre-trains a neural network to learn a differentiable simulator of the mathematics of the digital twin model which is then used to constrain the reconstruction of physiological measurements from data. This enables diverse applications from simulations of patient-specific models, which we demonstrate in our experiments in the setting of cardiac hemodynamics and the facilitation of in silico trials (for left-ventricular assistance devices, LVADs) and disease detection (mitral stenosis).


**Key Strengths**

* **Innovative methodology:** Our method is designed to be suitable to a challenging real world inverse problem. We combine the strengths of physics-informed models with self-supervised learning to address the challenge of modeling personalized digital twins without direct measurement. 
* **Non-invasive data:** Our approach is built with non-invasive indirect measurements. This has the potential to spur research into the development of medical digital twins suitable in a more diverse patient population.
* **Broad application:** The proposed framework allows for the simulation of detailed physiological parameters and interventions in any set up where a process can be modeled using ordinary differential equations. This generalizability will allow for further research for more widely applicable digital twins of various health systems and non-invasive modalities.
 
**Common Themes:**

* **Architecture and ablation studies:** We performed ablation studies to evaluate the impact of removing the physics-based model component and explored alternative architectures. The results showed that removing the physics-based model still allowed for comparable EF predictions, indicating that our model’s supervised learning capabilities were not compromised while enabling digital twins simulation. We explored various video feature extraction frameworks including LSTM and transformers besides 3DCNN and alternative differentiable ODE solvers finding that none performed as well in our experimental setting. We are open to suggestions to the reviewers of additional ablation studies to further validate our approach and set up but want to stress that our framework is not specific to any particular architecture. 
* **Generalizability of approach:** Our manuscript presents the example of the application of our methodology for modeling cardiovascular hemodynamics with non-invasive echocardiogram data. For this experiment, we selected parameters in a simplified dynamic lumped parameter model with the specific focus of evaluating patient cardiac pressure-volume loops. We evaluated the performance on two public echocardiogram datasets with diverse demographics and studied this experiment in depth to demonstrate a wide range of potential use cases for models developed in this framework. However, the problem set up and training are designed to be very general so that they might be used to develop digital twins for any system that can be modeled with ordinary differential equations using a mapping from a set of non-invasive observations. Our method is conceptual, aiming to foster data sharing and encourage broader applications in medical research.
* **Performance comparison and baselines:** We selected comparisons, specifically Physics-Informed Neural Networks (PINNs) and Neural ODE methods, to emphasize the fundamental differences in setup and to demonstrate differences in our surrogate model despite a comparable understanding of the underlying physics as state of the art methods. There are many variants of PINNs in the literature but not all are suitable to our set up. Our method is designed to approximate known and unknown components of an inverse problem. We propose a generative inverse problem approach that will work across parameter sets, which contrasts with the PINN and Neural ODE approaches that focus on learning physics for individual patients. Despite potential advantages of advanced PINN variants, our approach is tailored for capturing broader inverse relationships rather than optimizing for single-patient physics learning. We compared our approach to these baselines to illustrate similar performance while highlighting our method’s unique setup and goals. We remain open to incorporating additional relevant baselines if they align with our specific framework.

Meta-review: This manuscript proposes a novel method for creating patient-specific digital twins focused on cardiac hemodynamics. The general idea is how to infer from a non-invasive measure an invasive measure. The domain specific task aims to predict the physiological signal such as heart pumping blood from the recording of an echocardiogram image. The novelty of the method is to solve the associated inverse problem specifically for that patient, so that personalized predictions can be performed. The proposed method uses a two-steps Self Supervised Learning (SSL) approach that can be summarized by a step of pre-training and a step of fine tuning. A neural network is pretrained to learn from the data of a differentiable simulator the parameters of cardiac process. Then, another model is trained to reconstruct physiological measurements from non-invasive data while being constrained by physical equations learned during pretraining.

The rebuttal allowed the clarification of several aspect and the Reviewers agreed that the discussion was fruitful for a better understanding of the proposed method. No major concerns affect the soundness of this work after the rebuttal.

The controversial aspect of this work remains how should be perceived from the scientific community. The Authors believe that their contribution is general and the application to cardiac hemodynamics has only the purpose of an explanatory example. The Reviewers complain about this claim at least for two reasons: (i) the assumptions for the generalization are not clearly formulated, (ii) the successful application of this approach to other use cases is purely speculative and without empirical evidence. 
The proposed framework assumes at least a couple of premises: a simulator that encodes the relationship between a target variable x and an intermediate variable w, a collection of data where the annotation between an input variable y and the variable w is available (where x would be the invasive measure and y the non invasive measure). Even though this pattern can be recognized in several applications, the implementation is use case dependent.
For this reason this work could be considered for poster presentation.
Decision: Accept (poster)