A Physics-preserved Transfer Learning Method for Differential Equations
Authors: Hao-Ran Yang
Chuan-Xian Ren
Venue: NeurIPS 2025 poster ()
Keywords: Differential equations
Transfer learning
Optimal transport
Primary area: machine_learning_for_sciences

Abstract: While data-driven methods such as neural operator have achieved great success in solving differential equations (DEs), they suffer from domain shift problems caused by different learning environments (with data bias or equation changes), which can be alleviated by transfer learning (TL). However, existing TL methods adopted in DEs problems lack either  generalizability in general DEs problems or  physics preservation during training. In this work, we focus on a general transfer learning method that adaptively correct the domain shift and preserve physical relation within the equation. Mathematically, we characterize the data domain as product distribution and the essential problems as distribution bias and operator bias. A Physics-preserved Optimal Tensor Transport (POTT) method that simultaneously admits generalizability to common DEs and physics preservation of specific problem is proposed to adapt the data-driven model to target domain, utilizing the pushforward distribution induced by the POTT map. Extensive experiments in simulation and real-world datasets demonstrate the superior performance, generalizability and physics preservation of the proposed POTT method.

A Physics-preserved Transfer Learning Method for Differential Equations
Hao-Ran Yang
Chuan-Xian Ren' Sun Yat-Sen University Guangzhou, China rchuanx@mail. sysu. edu..
Sun Yat-Sen University Guangzhou, China yanghr266mai12- sysu edu.
Abstract
While data-driven methods such neural operator have achieved great success in solving differential equations (DEs), they suffer from domain shift problems caused by different learning environments (with data bias equation changes), which can be alleviated by transfer learning (TL). However, existing TL methods adopted in DEs problems lack either generalizability in general DEs problems physics preservation during training. In this work, we focus on a general transfer learning method that adaptively correct the domain shift and preserve physical relation within the equation. Mathematically. we characterize the data domain product distribution and the essential problems as distribution bias and oper- ator bias. A Physics-preserved preserved Optimal Tensor Transport (POTT) method that simultaneously admits generalizability to common DEs and physics preservation of specific problem is proposed to adapt the data-driven model to target domain, utilizing the pushforward distribution induced by the POTT map. Extensive experi- ments in simulation and real-world datasets demonstrate the superior performance, generalizability and physies preservation of the proposed POTT method.
Introduction
and industrial design 40 are Many scientific problems, such as climate forecasting 36 modeled by differential equations (DEs). In practice, DEs problems usually discretized and solved by numerical methods since analytic solutions are hard to obtain for most DEs. However, traditional numerical solvers struggle with expensive computation cost and poor generalization ability. Recently, dealing DEs with deep neural network has attracted extensive attention. These methods can be roughly divided into categories: physics-driven and data-driven. Optimizing neural networks with objective constructed by exact equations, physics-driven methods such as Physics-Informet Neural Networks (28 26 have great interpretability, but suffer from the poor generalization capability across equations and the hard requirement of exact formulation of DEs. In contrast, Data-driven methods such neural operator 24 22 typically take the alterable function in the equations as input data and solution function as output data. Their generalization capability are markedly improved as they can cope with a family of equations rather than one.
33
However, the performance of data-driven methods are highly dependent on identical assumption of training and testing environments. If the testing data comes from different distribution, model performance may degrade significantly. In practice, however, applying model to different data distributions is common requirement, e.g. from simulation data experiment data, cross-region model application. While it is often hard to collect sufficient data from a new data domain to train new model, transfer learning (TL) that aims transfer model from source domain with plenty of data
*Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Table 1: Simulation datasets used for experiments. Figures in D1. D2 and D3 are examples of output pairs from different domains in the transfer tasks. For 1-d curve plot, the filled regions represent the areas between the curve and the x-coordinate. For 2-d surface plot, the pixel value at each image pixel corresponds to the function value at the sampling point. Brighter color indicates larger value. Definition and more details about the subdomains datasets are provided in Appendix C.T
EQUATIONS
Di
D2
D3
BURGERS' EQUATION
M
U4 + ull Vilzzz
Ge : u(x,, 0) - u(x,
ADVECTION EQUATION
Ut + vul = O, Go u(2 0) + u(z,t)
DARCY FLOW (k(x)Vu(x)) 1, Go : k(x) + w(x)
to target domain with inadequate data, is widely adopted in real-world applications TL methods for DEs problems still unexplored.
39 However,
In this work, we carefully analyze the transfer learning settings DEs problems and modelize the essential problem as distribution bias and operator bias. Given such perspective, fully investigate the existing TL methods used in DEs problems. Technically, they can be summarized as three types. (1) Analytic methods [9] induce an analytic expression about model parameters to achieve adaptation with a few samples. Nevertheless, they are only feasible in very few problems with nice properties and hence not general methodology.
(2) Finetuning [31] the well-trained source model by target data. is widely-used in DEs problems with domain shift due to its simplicity. Nevertheless, when the amount of available target data is limited, directly correcting operator bias by finetuning is insufficient. (3) Domain Adaptation (DA) 344 methods developed in other areas such as computer vision. They typically align feature distributions of source and target domains and remove the domain-specific information, so the aligned feature together with the model trained from them are domain-invariant However, the physical relation of DEs may not necessarily be valid in the aligned feature space. Among these methods, analytic methods and finetuning directly correct the operator bias while the DA methods focus on the distribution bias correction. But all these methods have certain limitations. Therefore, a general model transfer method that can preserve the physical relation is worth exploring.
Our idea is characterize the target domain with physics preservation and then fully correct the Optimal Tensor Transport (POTT)
operator bias. Briefly speaking, we propose the method to learn physics-presd prresssepeeeervee optimal transport map between source and target domains. Then the target domain with the physical relations are characterized by the pushforward distribution, which enables comprehensive training for model transfer learning. Thus, the model's generalization performance on target domain can be largely improved even when only a small number of target samples are available for training. To encourage the POTT map characterize target distribution in way, we introduce a problem-specific physical regularization to the OT problem, which is derived from available physical prior of the problem. general cases without physical prior, the regularization term is formulated as relation between marginal pushforward distribution. Our contributions are summarized as follows:
A detailed analysis of transfer learning for DEs problems is presented, based on which we propose a feasible transfer learning paradigm that simultaneously admits generalizability to general DEs problems and physics preservation of specific problems.
We propose POTT method to adapt the data-driven model to target domain with the pushforward distribution induced by the POTT map. A dual optimization problem is formulated to explicitly solve the optimal map. The consistency property between the solution and the ideal optimal map is presented.
Extensive evaluation and analysis experiments on both simulation and real-world datasets are conducted. POTT shows superior performance on different types of equations with transfer tasks

---

of varying difficultiess. Intuitive visualization analysis further supports our discussion on the physics preservation POTT.
2 Preliminary
Data-driven methods for DEs problems. In DEs problems, data-driven methods aim to learn func- tional maps from data distributions. DeepONet (24] is proposed based on the universal approximation theorem 4 Then the MIONet [16] further extends DeepONet to problems with multiple input functions and Geoo-DDepOONe
[1] enables DeepONet to deal with parameterized 3D geometries.
Differently, FNO [22] is constructed in the insight of approximating integration in the Fourier domain. Geo-FNO 2T extends FNO to arbitrary geometries by domain deformations. F-FNO [32] enhances FNO by employing factorization in the Fourier domain. Recently, transformer have also been used to construct neural operators 19 14 20 351. Although having achieved great success, these data-driven methods induce common issue: they highly dependent identical assumption of training and testing environments. If the testing distribution differs, the performance of the neural operators will significantly degrade.
Transfer learning. Most of the transfer learning methods are proposed for Unsupervised Domain Adaptation (UDA) with classification task. They align the feature distributions of source and target domain by distribution discrepancy measurement [23]. domain adversarial learning [[] 3 etc. 38 further extend DA to regression settings with continuous variables. For
Recent methods (5] 27
DEs problems, analytic transfer methods ( are presents for specific equations; finetuning [37 31]
and DA methods 12 [
applied in various tasks. However, there isn't 't general physics- preeserdd transfer learning method developed for DEs problems.
Optimal transport (OT). OT has been quite popular in machine learning area 7 widely known OT problems are the Monge problem and the Kantorovich problem defined as follows:
6
The most
inf T P I clx, c(x,T(x)) (x)(x)
M(P3,
K(P*,
inf TEU(P= Seu((P.PP)..l PP) Ja
c(z, y) dr (x(())))
(2)
where c(x, y) denotes the cost of transporting x E Q2 to E Ω2, : - S2' denotes the transport map, Ts Ps denotes the pushforward distribution, and II(Ps, Po) denotes the of joint distributions with marginal Ps and Pa The Monge problem aims at transport map T that minimize the total transport cost, called the Monge map. However, usually the solution of the Monge problem does exist, so the relaxed Kantorovich poblem is more widely used. The Kantorovich problem can be solved as linear programming problem. With an entropic regularization added, it can be fastly computed via the Sinkhorn algorithm 77]. Moreover, if TT" takes the form π [id. T] 4 P E: II(P*,P*) P!), then T is the Monge map. However, these optimization method don't 't scale well to large scale data domain and can't t handle continuous probability distributions 330. To deal with these limitations, neural OT methods are developed (30 g 18. They typically train the neural network to directly approximate
the OT map via constructing objective function by various OT problems [00 13
3 Analysis and Motivation
3.1 Problem Formulation and Notations
Now we formally formulate the transfer learning settings for DEs problems. Consider two function space Dj. D,, with elements l S4 + R. 92 Denote the product spaces as D Di D. and Ω S4 x Q,, Suppose there exist physical relations within the product space D, which can be characterized as the following two forms:
F(k, u) = 0 (Equation form)
(3)
((k) = и (Operator form)
Obviously, relation Eq. 4 is the explicit form of the implicit operator mapping determined by Eq. 3
whose existence is theoretically guaranteed under some conditions such as the implicit function theorem. Once the operator Dx + D,, is solved, can predict the desired physics quantities
3

---

for group of k. However, directly solving the implicit function from Eq. (3) is often extremely difficult. In such cases, fitting by neural network with collected data set [kk, u)} provides practical way for numerical approximation, which is exactly the goal of data-driven methods. Here we slightly abuse the notations k and u to represent both functions and their discretized value vectors.
An essential limitation is that the learning of the operator network & depends heavily on the distribu- tion of collected data. Let P P Pd be two product distributions supported on the source and target domain D D' C D, respectively. Then the operator trained from them, denoted as Go and G+, are fact the approximations of G := G and G := Glp. When distribution shift occurs, the operator relation also differs. So the transfer learning problem for DEs can be modelized as
(* u) + PL(K, u), (Distribution bias)
(5)
(Operator bias)
In these situations, the model performance generally degrades if ς* is directly applied to D'. While collecting sufficient training data difficult in many applications scenarios, common requirement to transfer Ga to D' with a few target data available. Formally, given DO {(k?, ) y D' [(k), u²) )))} with n' < n", the task is to transfer source model Go to target domain D and approximate G" i.e. to correct the operator bias. An intuitive illustration is shown in Fig. 2
3.2 Methodology Analysis
Based on problem 5 existing transfer learning methods either directly correct the operator bias or indirectly correct the operator bias by aligning the feature distributions, all of which are subject to certain limitations.
Dg
Analytic methods directly correct the operator bias by deriving analytic expressions
D
g' = H
(6)
A(x)
=(x)
feuture
COD pred.
where H. denotes the ideal analytic formulation. Although exhibiting excellent interpretability, they are limited to problems with nice property and sufficient priors such the explicit forn of the equation. So they can only be applied to few problems with well-behaved equations.
Figure I: Visualization of DA method (COD) in
task D3 D2 on Darcy flow. The 1st and 2nd columns present the input k(z) and output u(z) sample pairs of D3 and D2. The 3rd column vi- sualizes their feature maps from the aligned dis- tributions, which are forced be analogous but lacks clear structure explicit physical meaning. It unclear whether they retain the correct physical information. The prediction shown in the 4th col- umn verifies that the physical structures of u are (7) not fully preserved.
Finetuning by target domain data directly cor- rects the operator bias by only further training the source model with collected target data:
G' min Crask Crask GD);
where Lrask denotes the task-specific training loss. does not actively and fully leverage the knowledge of source and target domain data. When the amount of available target data is limited, the predictions of target samples exhibit characteristics similar to the source samples since inadequate data is insufficient for model transfer, as discussed in Sec. 5
Distribution alignment methods from DA indirectly correct the operator bias by aligning the feature distributions. These methods typically aim to learn feature map and a corresponding feature space in which the distance between source and target feature distributions is minimized. Then predictor trained by the source domain features can be expected to perform well on target domain features:
g min dist(g .".0g"""""" P' g
g+ min Lrask(g (D*) U (D'): G
(8)
where dist(..)) :) denotes measurement of distribution discrepancy, is the learned feature map, I# Ps, g4 p are the pushforward feature distributions. DA method is purely data-driven without the need of physical priors it is general methodology that can be used in most scenarios. However,

---

F(k (r)) a²)=)
F(K =0
(k,,))
the POTT map
Pd (b,a) pi (R
G
g
Distribution bias
To
(k.u)
(2) Characterize P by P
g
g'
Operator bias
(0) Pretrain G
(3) Transfer & G
Figure 2: Illustration of POTT. Left: Illustration of problem formulation. The distribution bias leads to the bias of physics, i.e. the operator bias. The goal is to correct the operator bias. Right: POTT correct the operator bias by characterizing D transfer process, source model & pretrained with sufficient source data, (1) The POTT map To between source and target domain is learned. (2) The target distribution pt is characterized by the pushforward distribution P" (3) G is transferred to G with D' and Dr.
way. (0) Before the model
the two feature distributions are aligned by removing the domain specific knowledge so the domain invariant representations are obtained. In other words, the aligned feature distribution may lose some domain specific physical relations of both domain. As shown in Fig. [ the features of source and target samples are analogous but confused. It is unclear whether the physical relations are preserved. which is also indicated by the output of learned G'.
Motivation of POTT. Generally, directly correcting operator bias by finetuning only partially transfers with limited target data, while indirectly operator bias via feature distribution alignment may distort the physical relations of the DEs problem. Therefore, we propose to correct the operator bias by characterizing the target domain with physics preservation and then fully transferring & to D
Dr (D= D³, DD,,R, R), ğ' = min L U G';
(9)
where He characterizes D' by collected dataset D', Dr and the physical regularization R.
POTT Method
The major obstacle in Eq. 99) is construct the He. A practical way is to learn transformation T between P and pt, so the target distribution P can be characterized by the pushforward distribution = i.e. (-) = T(D:: D', R). Note that the exact corresponding relations between samples from pa and p is unknown, it impractical to train T by traditional supervised learning. In other words, T shall be trained via an unpaired sample transformation paradigm. In OT theory, an optimal transport map between of the OT problem does not need paired samples. Therefore, a natural idea is to model the ideal map by an OT map between ps and pt In the following sections we regard the desired map as the OT map T, distinguishing from map I" that is not necessary optimal.
distributions is the solution of an OT problem and the computation
4.1 Formulation of POTT
A brief review of OT problem is provided in Sec. 2 As our purpose is to characterize ps by pr = TT Ps, we focus on the Monge problem Eq, ① Moreover, the physical relation in DEs can be regarded as relation between Ps and P.. Thus, more reasonable perspective is to consider an OT problem between two product distributions, known as the Optimal Tensor Transport (OTT) problem. In this perspective,
propose the thysies-preserve optimal tensor transport (POTT) problem:
Definition 4.1 (POTT). Given two product distributions ps pt E PDDxX,, define the physics- preserved optimal tensor transport (POTT) problem
/ c ((ke u). T(k, u)) dp + R(T), D.
inf Ty P Pt
(10)
where T (T) (T] T p. R(T) R(T,, Tw)) ) is the physical regularization.
5

---

Specifically, when the pushforward distribution P" perfectly matches the target distribution Pt, the physical relation within automatically obtained. However, hard to achieve in practice and p should be regarded as approximation or disturbance of Pt. Then we expect p to approach p Ahysics-preserver way at least, which relies on the physical regularization R. The construction of R depends on the specific problem. Here we provide two basic ideas for example:
If some physical priors of the problem are available, R. can be derived from the priors. For example, when the weather forecasting problem is modelized as an advection partial differential
equation [33] the value preservation property f "u(x, t)dz = const, Vt, is strong inductive bias. Then the R can be formulated as the variance of the system's value:
R(T) var /f Tww l)xx),
(11)
Experiment of this problem shown in Sec.
For the general cases with physical priors available, we formulate R as the physical relation between the marginal distributions of Pr:
R(T) RT), m(G(k"), = =
(12)
where the .) can be any metric on Du. An alternative is the L2 norm in the vector space. In practice, the operator G can be substituted by 9 or G as an approximation.
Remark 4.2. While both PINNs-based methods and POTT emphasize the integration of physics into the learning process, they are not directly comparable. As described in Sec. [ PINNs-based methods rely on the explicit form of the equations to define the physics-informed loss. Consequently, they become inapplicable when the underlying equations are unknown only partially specified. In contrast, by leveraging the optimal transport framework and incorporating physical regularization, POTT can be considered combination of data-driven and physics-driven method and is applicable to broader range of problems.
4.2 Optimization and Analysis
Existing OTT methods 17 mainly focus on the discrete case of entropy-reeularized OTT and solve the optimization problem via the Sinkhorn algorithm, which is not suitable continuous OTT with physical regularization. Motivated by neural OT methods [30 18), we explicitly fit 'T by neural network and optimize it with the gradient of training loss. But Eq. I00 is a constrained optimization problem and it is challenging satisfy the constraint during the optimization process. Therefore, we introduce the Lagrange multiplier and reformulate Eq. (TO) to the unconstrained dual form.
sup inf upf [ c(k,, u), T(k, u)) f(T(k, u)) + XR(T) d.Pa + [ f(k, uddP'.
(13)
Optimization with gradient descent tends to converge to a saddle point (T*, Following previous work [10, the consistency between T and T is guaranteed.
Theorem 4.3 (Consistency). Suppose the dual problem Eq. 13) admits least one saddle point
solution, denoted as (T*, ff)) Let L be the objective of Eq. Then
the dual problem Eq. 13) equals the Kantorovich problem with physical regularization in terms of total cost, i.e. C( T .ft) [K(=K, ( P') + RT*).
T p = P then Eq. [3) degenerates to the dual form of the primal Monge problem Eq. (I. T is Monge map, i.e.
pt).
The proof of Thm 4.3 can be found in Appendix B Theoretically, if the Monge map exists, i.c. pr = Pc, then Pr automatically admits the physics contained in Pt. The solution of the Monge problem the desired OT map. However, as mentioned in Sec. in most cases the Monge map does not exists. Therefore, the saddle point T is not optimal solution of the physics-regularized Monge problem. this situation, [hmm4.3sstate. that Eq. (13) equals to pryssic-rreullarized Kantorovich problem in terms of total cost. Thus, the learned 7 can be regarded as compromise solution between the Monge problem and the Kantorovich problem. Importantly, the physical regularization encourages physics preservation during the training process of T, which is crucial for the approximation of Pt

---

Table Evaluation results of Burgers' equations.
Di + D2
D1
D3
D3
+ D2
AVERAGE
METHOD
50
100
50
100
50
100
50
100
TARGET ONLY
0.2142
0.1429
0.1173
0.0968
0.2142
0.1429
0.1819
0.1275
SRC+TGT
0.3960
0,3982
0.3486
0,3350
0.3145
0.2930
0.3530
0.3421
FINETUNING
0.2001
0.1191
0.1049
0.0801
0.1546
0.0938
0.1532
0.0977
TL-DEEPONET
0.1623
0.1182
0.1275
0.1127
0.1763
0.1436
0.1554
0.1248
0.1727
0.1145
0.1241
0.1099
0.1752
0.1393
0.1573
0.1212
COD
0.1713
0.1225
0.1288
0.1105
0.1818
0.1525
O. 0.1606
0.1285
0.1528
0.0965
0.0950
0.0705
0.1249
0.0757
0.1242
0.0809
POTT
#0.016
#000112
#000007
000000
#0.016
#:00199
20.013
20.013
Table 3: Evaluation results of Darcy flow.
TASK
D1 + D3
D2 - D>s
AVERAGE
METHOD
50
100
100
50
100
100
TARGET ONLY
0.1815
0.1122
0.3925
0,2893
0.3925
0.2893
0.3222
0.2303
SRC+TGT
0.7113
0.7600
0.1581
0.1409
0.3535
0.2381
0.4076
0.3797
FINETUNING
0.1426
0.0869
0.1556
0.1605
0,4693
0.3553
0.2558
0.2009
ET
0.1410
0.0805
0.1539
0.1481
0,4514
0.2842
0.2488
O. 0.1709
DARE-GRAM
0.1395
0.0805
0.1533
0.1441
0,4509
0.2842
0.2479
0.1696
COD
0.1367
0.0794
0.1527
0.1481
0.4437
0.2836
0.2444
0.1704
0.1362
0.0762
0.1397
0.1404
0.3527
0.2271
0.2095
0.1479
POTT
1000002
1.0.002
1:0.009
::0.006
00.025
1:00019
1:0.012
1:0 000009
In practice, we parametrize the map T, dual multiplier f and the operator & by neural networks To. fe and Gy with parameters denoted by 0, e and 77. Function variables & and are discretized into vectors. The overall objective of POTT method is
max min @
C(Kk uf), (K ut)) So(To ut) AR(TT)
Solk
(14)
min 7
Lrack (7)(k) u/) ) > (k)
where k.- = To, (kf), = To, (uē). Lrask is the task specific loss. À and are hyper-parameters. Physical regularization term R depends on the available physical priors, discussed in Sec. 4.1 A form of algorithm and the discussion of the computational cost are provided in Appendix C
5 Experiment
Experiments are conducted on both simulation and real-world datasets. For simulation datasets, three representative equations contained. For real-world datasets, consider the cross-region climate forecasting task. Implementation details and definition of RMSE are provided in Appendix
Benchmarks.
we take the 1-d Burgers' equation, 1-d space-time Advection equation, and 2-d Darcy Flow problem as our benchmarks. A brief introduction of these DEs problems can be found in Tab. I To simulate the domain shift, three different sub-domains for each equation, denoted as D, D2 and D3, are generated. We generate 1000 training samples for each domain of Burgers' equation and 2000 samples for Advection equation and Darcy Flow. To fully investigate effectiveness of transfer learning methods, we consider two scenarios that only 50 and 100 target data samples are available for model transfer.
Simulation dataset. Following previous works 22 25
7

---

Table 4: Evaluation results of Advection equations.
D1
D2
D2 + Dj
D3 - D2
AVERAGE
METHOD
50
100
50
100
50
100
50
100
TARGET ONLY
0.2162
0.1261
0.2585
0.1382
0.2162
0.1261
0.2303
0.1301
SRC+TGT
0.2347
0.1400
0.7299
0.4041
0.3969
0.1574
0.4538
0.2338
FINETUNING
0.0247
0.0143
0.2193
0.0891
0.1257
0.0723
0.1532
0.0977
TL-DEEPO
0.0587
0.0127
0.2365
0.1047
0.1534
0.0685
0.1495
0.0620
GARE-GRAM
0.0572
0.0121
0.2227
0.0805
0.1687
0.0700
0.1495
0.0542
COD
0.0530
0.0120
0.2252
0.0785
0.1593
0.0644
0.1458
0.0516
0.0207
0.0112
0.1872
0.0787
0.1016
0.0613
0.1032
0.0504
POTT
+00004
+:0000
#00002
#0.017
=0.015
00.007
#0.016
+0
z
12m
ul0
1200
1030
RMSE
800
600
400
6
12
10
24
36
6
12
18
24
36
6
12
18
24
36
12
18
24
30
12
10
24
36
Lead Time (hr)
Lead Time (hr)
Lead Time (hr)
Lead Time (hr)
Lead Time (hr)
Source-only
Finetuning
POTT
ClimaX larget
ClimODE larget
Figure 3; Comparision with (1) Source-only, source pretrained model, (2) Finetuning on target
domain, (3) ClimaX, SOTA climate forecasting model, trained with sufficient target data, (4)
ClimODE, the backbone model, trained with sufficient target data. The light area reflects the standard deviation of RMSE (L). Details are provided in Tab. 14
For all transfer tasks, we use 10 extra target domain samples for validation and 100 for testing, Note that the necessity of transfer learning depends on the extent of domain shift. For the transfer tasks with minor domain shift, source model can generalize well and additional transfer learning methods are unnecessary. Therefore, we only evaluate these methods in some hard tasks that need more proactive transfer. Details about data generation and how the domain shift between sub-domains is considered are provided in Tab. 10 and Tab. 9 IT 13 in Appendix C. Note that no equation-specific information (e.g., symbolic expressions or coefficients) is provided and only (k, u) data pairs are available during training, which mimics real-world conditions where governing equations are unknown. Therefore, the physical regularization R in Eq. is used for all these problems.
Real-world dataset. To better assess the potential of POTT in cross cross-domain application of data- driven methods, we evaluate POTT in the cross-region climate forecasting task. The preprocessed 5.625 resolution ERA5 dataset from WeatherBench [29] is used for evaluation and the SOTA ClimODE [33] is used as backbone model. Following 33 we consider 5 quantities as label variables: ground temperature (12m), atmospheric temperature (t), geopotential (z), and ground wind vector (u10
We use climate data on North America as source domain and the global climate data as target domain. For source domain, we use ten years of training data (2006-15) For target domain, only one year (2015) of training data available, while data of 2016 is used for validation and data of 2017-18 is used as testing data. More details are provided in Appendix C As the ClimODE is used as backbone model, we follow the idea that modelized the climate system advection equation and use the physical regularization in Eq. (IT) for POTT.
Comparision methods As discussed in Sec. 3.2 analytic methods are hard to apply in general data-driven DEs methods. So we compare POTT with finetuning and DA methods, including Finetuning on source pretrained model; Sre+Tgt, training from scratch with source and target data; TL.DDeppONet [2), a representative DA method proposed for DEs problems; DARE-GRAM (7 and COD 38 two SOTA DA methods proposed for DA regression problem with continuous variables. Since DARE-GRAM and COD are unsupervised DAR methods proposed for tasks in computer vision, we add the supervised target loss to them for faimess.
8

---

k(x)
x(x)
POTT pred.
F.T. pred.
COD pred.
POTT em
FT. er....
COD en
Figure 4: Visualization of G. Darcy Flow. The value at each image pixel represents the function value at the sampling point. Brighter colors (yellow) indicate higher values. Columns 1-2 show the input-output function pairs from the target domain. Columns 3-5 show the output of POTT, finetuning (F.T.) and COD. Columns 6-8 display the prediction errors relative to the ground truth u(x).
Evaluation results. For simulation datasets, average rMSE of three times repeated experiments are reported. For climate forecasting, the mean and standard deviation of latitude-weighte RMSE is reported. Details about the evaluation metrics are shown in Appendix
Simulation datasets. As shown in Tab. 3 POTT outperform finetuning and DA methods in most tasks. tasks with severe domain shift such as D2 - D3 of Darcy flow, the performances of existing methods are not satisfactory, while POTT reduces the relative error by nearly 25% (from 0.4693 to 0.3527) with only 50 target smaples available, and reduces the relative error by 36.08% (from 0.3553 to 0.2271) with 100 target samples. In simpler tasks D2 - 'D1 and D + D3. POTT still reduce the relative егrоr compared to finetuning and DAR methods in every task. Results of Burgers' equation and Advection equation are provided in Tab. 2 and Tab.
Climate forecasting. We only compare POTT with finetuning method because the DA methods are not easily applicable for the model architecture of ClimODE. As shown in Fig. 3 directly applying ClimODE trained on North America to global forecasting results in distinct prediction error. The model performances
improved by finetuning, and POTT further reduce the prediction error, approaching the ClimODE model trained with sufficient target data. Note that the prediction error of POTT is quite closed to ClimaX model trained with sufficient target data, which also a SOTA model climate forecasting. Such experiment demonstrates the potential of POTT general model transfer method for DEs problems and real-world applications.
Visualization analysis of prediction. Fig. illustrates the outputs and error maps for the target sample predicted by POTT, finetuning, and COD on the Darcy D3 - D2 task with 100 target samples. (1) shown columns 3 5. despite only 100 target samples are available for training, the shape and the variation trend of outputs predicted by POTT are consistent with the ground truth. In contrast, predictions of finetuning indicate that the transferred model fails learn the right shape and variation trend. Predictions of COD are globally consistent with ground truth, but the transferred model fails to correctly predict the large value areas. (2) the error maps shown columns 6 8, the bright areas in the error maps of POTT are the smallest among the three methods. Especially, the error maps of finetuning shown 7th column clearly exhibit the characteristics of source domain distribution. i.c., the distinct triangular patterns, supporting the discussions in Sec. 3.2
Ablation analysis of physical regularization. To investigate the effect of the physical regular- ization Eq. (10). we implement an ablation analysis on the task D2 + D2 of Darey flow. As shown in Fig. 5 the outputs OTT roughly resemble the ground truth Mtgt in the regions of large values, but the triangular structure is not preserved, indicating the loss of certain physical
Uyre
Musi
мPOтT
MOTT
Figure 5: Visualization of POTT and OTT.

---

relations. In contrast, with physical regularization, the outputs of POTT exhibit consisteney in both the regions of large values and the triangular structure, verifying the preservation of physics.
To more intuitively demonstrate the effect of physical regularization, we present the numeri- cal results of &. trained using the outputs of OTT and POTT, as shown in Tab. [S] It is observed that POTT achieves lower prediction error than OTT, indicating that the outputs of POTT better characterize the target distribution in pressrsvd manner. Furthermore, both OTT and POTT outperform vanilla finetuning.
Finetuning 0.0575
POTT w/o R
POTT 0.0496
2) D2
0.0543
Table 5: Results of the ablation analysis.
Extreme few-shot settings. In extreme few-shot settings (e.g., less than 10 target sam- ples), both finetuning and POTT indeed face signifi- cant challenges due to the risk of overfitting. From a citrrbution-matchiin perspective, fitting a meaningful target distribution with only a few samples is fundamentally ill-posed and prone to instability. This limitation applies broadly to all distribution- based methods. To empirically evaluate this, we conduct a scale-down experiment on the Darcy flow task D2 + D3 under various numbers of labeled target samples n². As shown in Tab. 6
n²
100
50
20
10
5
I
Tgt. only Finetuning POTT
0.3943
0.3925
0,4130
0.4897
0,5107
0.5568
0.3553
0.4693
0.4866
0.5368
0,5521
0.6282
0.2271
0.3527
0.4126
0.5257
0.5364
0.6359
Table 6: Results of scale-down experiment.
As n'' decreases, all methods degrade in performance.
When n' > 20, POTT still provides advantages over both finetuning and training from scratch.
When n' > 10, both POTT and finetuning begin to overfit, and performance starts to deteriorate significantly.
At the extreme case of n' = 1. POTT fails entirely, the transport map becomes unreliable and may even hurt the learning of the target predictor due to inaccurate distribution matching.
These results clarify that POTT most effective in the moderate few-shot regime, but may not be applicable when labeled target data is extremely scarce, which aligns with many real- world cases in scientific machine learning where acquiring a small number of high-quality labels is fcasible.
Conclusion
In this work, we studied the domain shift issue in DEs problems. The essential problem is mod- eled as distribution bias and operator bias. Then detailedly analyzed existing transfer learning methods used in DEs problems and propose a feasible POTT method that simultaneously admits generalizability to common DEs and physics preservation of specific problem. Based on the the availability of physical prior, two forms of realization of are introduced to encourage the POTT map to characterize target distribution in a chysics-preserved way. A dual optimization problem and the consistency property are formulated to explicitly solve the optimal map. Extensive evaluation and analysis experiments on both simulation and real-world datasets with varying difficulties validate the effectiveness of POTT for fata-driven methods in DEs problems.
Limitations and future works. (1) The core of POTT lies in characterizing target distribution with limited samples. However, similar to other distribution-based methods, POTT''s efficacy diminishes when the quantity of available target data is extremely small, rendering its potential in one-shot or few-shot scenarios. Moreover, when abundant target data are accessible, finetuning or even training from scratch sufficient for cross-domain application. Therefore, POTT is more valuable when the amount of target sample is small, yet not extremely small, as shown in Sec. and Sec. D (2) The learning of POTT relies on the min-max optimization, which complex and costly for models with large-scale parameters. The resolution of the experimental data is relatively low, thereby necessitating less complex model. However, in DEs problems and application scenarios that require high resolution, a larger model is needed to fit POTT. To address this issue, one idea is to leverage the well-developed multimodal large models to reduce the training load of the method, which will further enhance the practicality and broad applicability of this method. We believe this will contribute to the generalizability and universality of the scientific and technological achievements, such as the cross-regional climate forecasting tried in this paper.
10

---

Acknowledgments
This work is supported in part by National Key R&D Program of China (2024YFA1011900 1900), Na- tional Natural Science Foundation of China (Grant No. 62376291), Guangdong Basic and Applied Basic Research Foundation (2023B1515020004) Science and Technology Program of Guangzhou (2024A04J6413) and the Fundamental Research Funds for the Central Universities, Sun Yat-sen University (24xkjc013).
References
[1] Arip Asadulaev, Alexander Korotin, Vage Egiazarian, Petr Mokrov, and Evgeny Burnaev. Neural optimal transport with general cost functionals. In ICLR, 2024.
[2] Nikolas Borrel-Jensen, Somdatta Goswami, Allan P. Engsig- Karup, George Em Karniadakis, and Cheol-Ho Jeong. Sound propagation realistic interactive 3d scenes with parameterized sources using deep neural operators. Proceedings of the National Academy of Sciences, 2024.
[3] Lin Chen, Huaian Chen, Zhixiang Wei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. Reusing the task-specific classifier as a discriminator: Discriminator-Free adversarial domain adaptation. In CVPR, 2022.
[4] Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions Neural Networks, 1995.
[5] Xinyang Chen, Sinan Wang, Jianmin Wang, and Mingsheng Long. Representation subspace distance for domain adaptation regression. In ICML, 2021.
[6] Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016.
[7] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In NeurIPS, 2013.
[8] Max Daniels, Tyler Maunu, and Paul Hand. Score-based generative neural networks for large-scale optimal transport. NeurlPS, volume 34, pages 12955-12965, 2021.
[9] Shaan Desai, Marios Mattheakis, Hayden Joy, Pavlos Protopapas, and Stephen J. Roberts. One- shot transfer learning of physics-informed neural networks. In ICML Al for Science Workshop, 2022.
[10] Jaaojia Fan, Shu Liu, Shaojun Ma, Hao-Min Zhou, and Yongxin Chen. Neural monge map estimation and its applications. Transactions on Machine Learning Research, 2023.
[11] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Leurning Research, 2016.
[12] Somdatta Goswami, Katiana Kontolati, Michael Shields, and George Em Karniadakis. Deep transfer operator learning for partial differential equations under conditional shift. Nature Machine Intelligence, 2022.
[13] Nikita Gushchin, Alexander Kolesov, Alexander Korotin, Dmitry P Vetrov, and Evgeny Burnaev. Entropic neural optimal transport via diffusion processes. In NeurIPS, volume 36, pages 75517-75544, 2023.
[14] Zhongkai Hao. Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In ICML., 2023.
[15] Junyan He, Seid Koric, Diab Abueidda, Ali Najafi, and Iwona Jasiuk. Geom-deeponet: A point-cloud-ban deep operator network for field predictions on 3d parameterized geometries. Computer Methods in Applied Mechanics and Engineering, 2024.
11

---

[16] Pengzhan Jin, Shuai Meng, and Lu Lu. Mionet: Learning multiple- operators via tensor product. SIAM Journal on Scientific Computing, 2022.
[17] Tanguy Kerdoncuff, Rémi Emonet, Michaël Perrot, and Mare Sebban. Optimal tensor transport. In AAAI, 2022.
[18] Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Neural optimal transport. In ICLR, 2023.
[19] Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations" operator learning. Transactions on Machine Learning Research, 2023.
[20] Zijie Li. Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. In NeurIPS, 2023.
[21] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 2023.
[22] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In ICLR, 2021.
[23] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks, In ICML, 2015.
[24] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 2021.
[25] Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 2022.
[26] Yiming Meng, Ruikun Zhou, Amartya Mukherjee, Maxwell Fitzsimmons, Christopher Song, and Jun Liu. Physics-informe neural network policy iteration: Algorithms, convergence, and verification. In ICML, 2024.
[27] Ismail Nejjar, Qin Wang, and Olga Fink. Dare-gram: -ram: Unsupervised domain adaptation regres- sion by aligning inverse gram matrices. In CVPR, 2023.
[28] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-infone informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019.
[29] Stephan Rasp, Peter D Dueben, Sebastian Scher, Jonathan A Weyn, Soukayna Mouatadid, and Nils Thuerey. Weatherbench: a benchmark data set for data-driven weather forecasting. Journal of Advances in Modeling Earth Systems, 12(1) )=2200MS0002203, 2020.
[30] Vivien Seguy, Bharath Bhushan Damodaran, Remi Flamary, Nicolas Courty, Antoine Rolet, and Mathieu Blondel. Large scale optimal transport and mapping estimation. In ICLR, 2018.
[31] Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior. In NeurlPS, 2023.
[32] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In ICLR, 2023.
[33] Yogesh Verma, Markus Heinonen, and Vikas Garg. ClimODE: Climate and weather forecasting with physics-informed neural ODEs. In ICLR, 2024.
12

---

[34] Yuchen Wang, Can Xiong, Changjiang Ju, Genke Yang, Yu-wang Chen, and Xiaotian Yu. A deep transfer operator learning method for temperature field reconstruction in a lithium-ion battery pack. IEEE Transactions on Industrial Informatics, 2024.
[35] Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for PDEs on general geometries. In JCML, 2024.
[36] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with unified deep model. Nature Machine Intelligence, 2023.
[37] Wuzhe Xu, Yulong Lu, and Li Wang. Transfer learning enhanced deeponet for long- time prediction of evolution equations. In AAAI, 2023.
[38] Hao-Ran Yang, Chuan- Ren, and You-Wey Wei Luo. Cod: Learning conditional invariant representation for domain adaptation regression. In ECCV, 2025.
[39] Yuchen Zhang, Mingsheng Long, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael Jordan, and Jianmin Wang. Skilful nowcasting of extreme precipitation with nowcastnet. Nature, 2023.
[40] Tingtao Zhou, Xuan Wan, Daniel Zhengyu Huang. Zongyi Li, Zhiwei Peng, Anima Anandkumar, John F. Brady, Paul W. Sternberg, and Chiara Daraio. Ai-aided geometric design of anti-infection catheters. Science Advances, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The cliams made in the abstract and the end of the introduction accurately summarize the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The limitations about the method and the experiments, influence factor of the performance, and the future works are discussed in Sec. 6
The discussion on computational
efficiency of the algorithm is provided in Appendix B.3
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: The full set of assumptions and proof are provided in Appendix B And an intuitive understanding of the Thm. 4.3 is provided right after the theorem.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All details about the data generation and implementation are provided in Sec. 5 and Appendix Some experiment settings are based on previous works in different areas, we have made clear instruction in these sections.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: As our work is about general model transfer method for DEs problems, the reproducibility can be guaranteed by the algorithm description and implementation details
provided in Sec. Appendix B.3 and Appendix C
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttpss//niis.co cc/ public/guides//ane Soaaedeee/DoeeSuuusssiiinoonnni
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ liiaas.ccc/ublic/giicee http:/guiiese///eeee
www.ibmmsssiooPooo ssioncl for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental settings and details are provided in Sec. and Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: The std of results are provided.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We indicate the compute resources in Appendix C.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and have made sure the reascarch conform the Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The potential positive societal impacts discussed in Sec. 6 As our work is a general model transfer method, the potential negative socictal impacts depends on the specific transferred data-driven model and its applications.
Guidelines:
17

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work is general model transfer method, the potential risks depends on the specific model that use this method.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The original papers that produced the code package and datasets are properly credited and cited in Sec. (5c.|5)am and Appendix All codes and datasets used for both simulation and real-world datasets takes the MIT License.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
18

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, palasets.papersstt paperswi thcode. com con/dataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No assets are released in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy ottp:e///eerrrpoeccco//oeeeerere for what should or should not be described.
20

---

A
Notations
The notations appear in this paper summarized follows:
Table 7: Notations.
Notation
Description
D DR x Du {(k, u)}
General function
Ω 94 x f2n
Domain of function
k: = k(x)
function defined on S2 function defined on Q Differential Equation
" == ((x) F(x; k,
D: Di
Operator map from D3 D,,
u(
Product probability function defined on function set
p(k,u) u)
Product probability density function function set
Pp
Set of probability functions defined on D
D'
Soure domain; subset of 'D
D
Target domain: a subset Operator relation on D Operator relation D Probability function D Probability function on D Function map between D and D Ideal solution of optimal transport problem Monge problem between probability P and Kantorovich problem between probability P and p'
G
G'
P
p
T: D D T
M(P", P()
K(P*, P R(T)
Physical regularization
m()))
Metric defined on D.,
(P", Pt)
Monge problem with physical regularization
K phyy (Pa p() cl.))
Kantorovich problem with physical regularization
Cost function in OT problem
Lagrange multiplier
Objective function of optimization problem Saddle point solution of dual problem
(1"
Approximated operator on D
G
Approximated operator on D Approximation of T
T
we slightly abuse the notations & and u to represent both functions and their discretized value vectors. The superscript denotes the domain. The subscript & or denotes the projection of the original product space or distribution.
B Theory and method
B.1
Derivation of dual formula
Given POTT problem
inf pt= pt / c((k, u). T(k, u)) dP +R()),
(15)
where =
(T,, Ta), Te = Tlp, Tu = Tlo,, we reorganize it as constrained optimization
problem:
inf / c c(k, u), T(k, u)) dPn R(T)
(16)
T
s.t.T P pt
(17)
Following the dual optimization theory, we introduce the Lagrange multiplier f to construct the Lagrange function:
C(T, / xm. c(kk,)), dP
dp XR(T) // J f(k, u(k,,)d(Pt Po). (18)
21

---

As the physical regularization regularized the pushforward distributions P = =T T* it can be considered with the source distribution. Then comes
Ps,
f)
Sux c((k, (), ((T(k,)) XR(T)dP" SuxR
udk, (19)
And the dual problem of Eq. 15) is
sup supinf inf C(T, f),
(20)
f
which is exactly Eq.
13
B.2 Proof of Thm. 4.3
prove Thm. 4.3 based on previous work [10]
Theorem B.1 (Consistency). Suppose the dual problem Eq. (33 admits least one saddle point
solution, denoted as (T* ..)) Let L be the objective of Eq. 13 Then
the dual problem Eq. (13) equals the Kantorovich problem with physical regularization in terms of total cost, i.e. L( C(pa, T", = ((P,, P') + R(T").
i ps = Pt, then Eq. (13) degenerates to the dual form of the primal Monge problem Eq. I T is a Monge map, i.e. S(P"TT",TT) M(Ps,PP
Proof. (1) Let kJ = (k"),, -To (u*), the inner optimization problem can be formulated as inf ()
=inf/ inf / c <((x,, ((hk",")) TKk,,((",,"")) f(()))(((())))))) AR(T) . F(W,W))P
- I (E")T)
(f((".5)) -ce -c)- (I)", (E", (E",)))) + AR(T))aPP + +f (( (( - f pa-((k, u))
(21)
sup
where
for (k= -²)= sup (f(E" (") |e ((k",")) (E), (()) + AR(T))) (et,T))
(22)
is the c-transform of the physics-regularize Kantorovich dual problem. Then the optimization problem Eq. (13) becomes
sup F(Kt, - - (k",
(23)
which is exactly the physics-regularized Kantorovich problem.
Therefore, if (f) f*) the saddle point solution of Eq. (13), then f* an optimal solution of Eq. 23 C(Ps, T " f*) K(P=, PO) + R(T*), which verifies the first assertion of the theorem.
(2) The saddle point (T*, (^) satisfy
(k" u²) E argmaz(ee.c - (k*, w²) (T* (k*, le((k' ("),""""),, T" (k",, w²)) + XR(T))
(c(("" "f", XR(T))
(24)
(25)
where fke--(k,, u²) ) = sup (r) (f" - [c((k" (5", (")) + XR(T)))
With condition T3 p pt. the pushforward distribution p pt then from the construction of RT)), we have RT) 0. Thus Eq. (13) degenerates to
(
suping inf f T
c((k, u), T(k, T(kk, w)) f(T u))dd=
f(k, udk,
(26)
Dxx0
22

---

which is exactly the dual form of the primal Monge problem. Then we have
c ((k , T (k", u²)) dp
/ (T" u²) dd)) / (k",
u ddP
Jn
112
[. M' w' / pes (e5--h" uddd
(27)
I (* - - wd)d
S cl(k",
" ,u",, (k', v²)) dr,, Vo II(P", Pt)
Take infimum on both sides of the inequation, we obtain
I
inf c((k", (((K",")) u²),," (k", ,u')) dp=
T In
I c ((W,, u²), (W,, u²)) dm
inf
(28)
Jaxa
< 1/8 c ((h", T(k", u²)) dPs,
where T' is any map that satisfies (Id, T) = = II(P" Pt). Therefore, the solution of the Monge problem exists and T is the Monge map.
B.3 Algorithm analysis
Algorithm 1: Optimization of POTT
Input: source data D., pretrained model Gy, target data Dj; 2 Initialize Ta.
for Nn sleps do Freeze d update minimize the first objective of Eq. (4) for N12 steps;
Freeze 0, update & maximize the first objective of Eq. with 0;
6 end
for N₂ steps do Freeze and o, update to minimize the second objective in Eq. 14
end
10 Output: G, as approximation of G..
Thus, the entire training process of POTT requires O((N1 -N1 NN) + C, +G)) operations. where Co, Cn, Cg are model parameters, B is the batchsize, N12 is typically set to 10. As the dataset size grows larger, N1 and N₂ should be larger, and the model size of Ta and fø also grow. In summary, the computational efficiency of POTT is similar to other neural OT methods.
C Experiment details
C.1 Simulation datasets
The relative Mean Square Error rM SMSE = |[Upred Ugt /1// ugell is reported for evaluation, where Ugt denotes the ground truth of output
Simulation datasets used for evaluation are generated as follows:
C.1.1 Burgers' equation
Considering the 1-D Burgers' equation on unit torus:
+ ull Vilam (0. E (0.1).
(29)
23

---

we aim to learn the operator mapping the initial condition to the solution funciton at time one, i.c. Go : Up = u(z, 0) u(z 1). As shown Tab. g we differ the generation of ug and parameter to construct different domains:
Table 8: Generation of ug and parameter settings 1-d Burgers' equation.
BUB-DOMAIN
DESCRIPTION
D1 up ~ (-
0.01
D2 uo ~ (O.2,, (0.22,99((-A))) 721) 2.5) V 0.002
D3 Uo
.V.0., 625 25 2.0), 0.004
The N denotes the normal distribution. The resolution of x-axis is 1024.
We assess the domain shift between these domains in rather straightforward way. Specifically, regard the data-driven model trained with sufficient target data as Oracle, and compare the prediction error between the finetuning method trained with 100 target data and the Oracle. Tasks with lower gap are considered as simpler than tasks with larger gap. As shown in Tab, 9 the prediction gap in tasks D2 Dj, D2 + D3, D3 + D1 are smaller than the other three, which means models are easier to transfer in these tasks. So we only conduct comparision experiment in tasks D1 - D2, D2 + Dy and D3- + D2.
Table 9: Evaluation tasks' difficulties of Burgers' equation.
METHOD
Dj + D2
D1 + D3
D2 + D1
D2 + D3
D3 + D2
D3 + D2
FINETUNING
0.1191
0.0801
0.0381
0.0786 0.0403
0.0252
0.0938
ORACLE
0.0402
0.0403
0.0140
0.0140
0.0402
As shown in Tab. 2 the improvement of POTT compared to finetuning is substantial. When the amount of target data is only 50, POTT reduced the relative error by 23.64% (from 0.2001 to 0. 1528) in task Dj - D2 and by 19.21% (from 1546 to 0. 1244) in task D3 + D2- When the amount of target data is 100, although the performance of finetuning greatly improves, POTT still largely reduces the relative error by 18.98% and 19.30% in task D + D2 and D3 - D2. In the relatively simple task D, Da. although finetuning already achieves satisfactory results, POTT can still reduce the relative error by about 10%, while TL-DeepONet, DARE-GRAM and COD even caused negative transfer and result in larger relative error.
C.1.2 Darey flow
The 2-d Darcy flow takes the form
(k(x)Vu(x)) = 1, E 00. 1] x [0 u(x) 0, x 0(00 1] x [0, 1])
(30)
We aim learn the operator mapping the diffusion coefficient k(x:) the solution function u(x). i.e. Go : k(Ø) - u(z). We use the leading 100 terms in a truncated Karhunen - Loève (KI.
expansion for Gaussian process with mean and covariance kernel K((x:) generate a(x), and construct different function domains by differ the kernel ((x,x"), as shown in Tab. 10
The Pequore denotes square with vertice on {0, 0), (0, 1), (1, 0), (1, 1), in 00, 1] [0. Ptriongle
denotes a triangle with vertice on {(0, 0), (0, 1), (0.5, 1)} in [0, 1] x [0, 1]. The resolution of x [0, 1] [0, 1] is 64 x 64. Similarly, the difficulties of transfer tasks in Sec. 5 we conduct experiments in the hard tasks D2 + D1, D1 - D3 and D2 - DD. Results are provided in Tab. 3
shown in Tab. 11 As discussed
C.1.3 Advection equation
The Advection equation takes the form
Ue + Dug = 0, x E (0, 1). (0,
(31)
24

---

Table 10: Generation of a(x) in Darcy flow.
SUB-DOMAIN
DESCRIPTION
D1
X((x,²') expl |
Tu Rsquare
D2
K(x,x') exp( ||-- 12
) Qu -triiangte
X((x,2') eap( x 11? Sh Ruyuore
Da
Table 11: Evaluation of tasks' difficulties of Darcy flow.
METHOD
D1 + D2
Di + D3
D2 + D1
D2 + D3
Da + D1
D3 + D2
FINETUNING
0.0429
0.1605
0.0869
0.3553 0.0682
0.0469 0.0153
0.0598
ORACLE
0.0111
0.0682
0.0153
0.0111
We aim to learn the operator mapping the initial condition to the solution funciton at continuous time set [0, 1], i.e. Go Mi u(z, 0) u(x; t). As shown in Tab. [2] we differ the function types and generation process of uh and parameter v to construct different domains:
Table 12: Generation of uo and parameter settings in Advection equation.
SUB-DOMAIN
DESCRIPTION
Di
(x(x) az + ba +c, d uo(x) az ba cr d. E U40
D2
= uo(x) abx + c), a U(0, CE
D3
The ul denotes the uniform distribution. The resolution of x-axis and t-axis are 100 and 50, respectively. Similarly, the difficulties of transfer tasks are shown in Tab. 13 For this equation, we select three easier tasks to explore the effectiveness of POTT in easy tasks, i.e., the D2 D2, D: - D2 and
Dy + D2. As shown in Tab. 4 although the performance of finetuning is satisfying, POTT can further reduce the prediction eтrоr to a lower level.
C.2 Real-world datasets
We use the preprocessed version ERA5 from WeatherBench 29 as real-world datasets for climate forecasting task. We utilize the ClimODE (33 as backbone model and conduct experiment based on their settings and codes, which are all available from their paper. Note that we only report the laiinuee-weighted RMSE for evaluation, since the Anomaly Correlation Coefficient (ACC) reported 33 rather closed for most methods comparision. The definition of the latinude-weighted RMSE is
H
W
1 N Σ HW ΣΣ ahh)(ugr Upred)?
RMSE
where a(h) = cos(h)/ / ΣN cos(h') is the latitude weight, Ugt and upred are ground truth and model prediciton respectively. Lower laiitue--eighted RMSE values indicate better model perfor- mance in capturing spatial or climate patterns. Visualized results and analysis are provided in Sec. 5
details results are provided in Tab. 14
C.3 Implementation details
Simulation experiment. test the generalizability of POTT with different models, we employed different backbones various datasets. On the Burgers' equation dataset, G, is parametrized as 1-d Fourier Neural Operator (FNO) model, To is an operator network composed of two fully connected networks (FCN), and is FCN. On the Advection equation and Darcy flow datasets, G, adopt
25

---

Table 13: Evaluation of tasks' difficulties of Advection equation.
METHOD
D D2
D1 D3
D2
D2 D3
D3 D1
D3 D2
FINETUNING
0.0143
0.0795 0.0134
0.0891 0.0252
0.0777
0.2226 0.0252
0.0723
ORACLE
0.0123
0.0134
0.0153
Table 14: Latitude weighted RMSE(L) results of global forecasting on ERA5 dataset. Oracle denotes the model trained with sufficient target data.
Lead-Time (hours)
ClimODE
ClimaX
Source-only
Finetuning
POTT
Oracle
Oracle
265.71 14.76
216.77 + 14.58
196.33 14,89
102.9 9.3
247.5
12
405.83 + 27.34
295.89 + 22.92
=
134.8 + 162.3
265.3
18
568.39 +: 35.93
381.6 1: 29.34
325.7 32.28
12.7 + 14.4
319.8
24
723.61 +: 43.37
445.45 + 35.35
374.68
39.66
193.4 16.3
364.9
1127.46 +: 58.67
598.61 44.72
506.43
51.64
259.6 22.3
455.0
1.81 1.81 0.07
1.63 1 0.07
1.36
1.16
1.64
12
2.72 + 0.14
2.17 + 0.09
1.85
+
1.77
18
3.51 0.19
2.1
0.19
1.16
1.93
4.51
0.11
1.11
0.18
2.17
36
6.96
0.14
0.19
1.75 + 0.26
2.49
0.21
0.23
2.6
2.02
12
9.83 0.24
0.25
3.2+
1.45 0.10
2.26
12m
9.21
3.14
0.2
1.43 +
9.69 - 0.36
3.01
2.31
1.00 1 0.09
2.37
36
10.91 1: 0.67
3.22
3.28
1.70 1 0.15
2.87
2.22 0.11
2.19
2.11
141
1.58
3.09 0.15
2.112
0.13
1 0.09
1.96
u10
18
3.8 1: 0.16
3.11
0.13
1.97 0.11
2.24
24
0.17
0.13
0.15
2.01
2.49
36
0.2.
4.05 0.15
0.17
0
2.98
2.15
0.13
2.14
1.53
1.60
12
3.4
2.88 0.14
0.16
1.11
1.97
v10
18
4.03:00:20
0.15
0.17
1.96
2.26
24
4.58
0.25
0.15
2.86
2.14
2.48
5.72
0.25
4.21 0.18
3.36 = 0.22
0.24
2.98
2-d DeepONet model, T3 has structure similar to Gy, and fo is convolutional neural network (CNN). We use Adam as optimizer and the learning rate is le for all tasks. The learning rate of the backbone of G, is ten times smaller than the last two layers, which is widely-used technique in transfer learning. A cosine annealing strategy is adopted for learning rate of G Details of model architectures and data generation can be found in codes provided by [2 25).
Climate forecasting. The architecture and training details of the data-driven model G, are consistent with the codes provided by [3). The architecture and training procedures of To and fo are the same with that in the simulation experiment.
All experiments are conducted on single 16GB NVIDIA 4080 device.
D Further discussion
D.1 Practical value of POTT
As the absolute errors of POTT still too large for practical applications, we discuss the practical value of POTT compared to finetuning with more target data.
POTT is proposed performance under limited supervision. As standard in transfer learning literature, our primary focus is the relative gain over basclines such as source-only and finetuning, rather
transfer learning framework designed to improve
26

---

D1 D3
200
400
600
800
1000
FT
0.1482
0.1391
0. 1001
0.0928
0.0867
POTT
0.1309
0.1
0.0899
0.0817
0.0792
Table 15: Results in the extreme few-shot settings.
D2 + D3
200
400
600
800
1000
FT
0.2339
0.1668
0.
0.1055
0.0947
POTT
0.2086
0,1487
0007
0.0914
0.0834
Table 16: Results in the extreme few-shot settings.
than the absolute error, which also depends the model's expressiveness. In experiments, all methods use the same architecture to ensure a fair comparison.
Similar evaluation protocols are widely adopted in transfer learning research. For example, in domain adaptation tasks DomainNet using ResNet-101, SOTA methods achieve only about 40% absolute accuracy. Nevertheless, they are considered effective due to their consistent improvement over source-only baselines (about 25%). This demonstrates that the relative improvement a well-accepted metric for evaluating transfer methods.
seems that the errors of about 0.15 and 0.21 in Darcy flow with 50 and 100 target samples are still too large for practical applications. However, this is mainly due to the limited capacity of the baseline model. Even with 2000 labeled samples in domain D3, the fully supervised model reaches an еπrоr of about 0.07. This highlights the inherent difficulty of the task. To investigate further, we conducted scaling experiments as shown in Tab. 15 and Tab. 16 The results confirm that while increasing the amount of labeled data does improve performance, POTT can achieve comparable accuracy with significantly fewer samples. For instance, POTT with 600 800 samples matches finetuning with 1000 samples, which is very close to the oracle performance achieved by the backbone model ( (0.07).
In many practical applications such as high-resolution simulations, chaotic weather systems, or medical diagnostics, collecting a moderate number of labeled samples is feasible, but obtaining large-scale annotations remains prohibitively expensive. In such scenarios, finetuning alone may fall short, and incorporating principled transfer methods like POTT can effectively compensate for the data scareity.
In summary, while achieving very low absolute error (e.g., <0.05) may require more powerful models or larger datasets, our results clearly show that POTT significantly improves performance in the low-data regime, reducing the required number of labeled target samples to achieve similar accuracy as finetuning. This highlights its practical value in scenarios where labeled data is limited but not extremely scarce a common situation in many scientific and engineering domains.
D.2 Relations between the reported evaluation metric and the residuals.
Whlie the evaluation metric used in our experiments is the relative L2 error, it is closely related the residuals of the governing equations, which is widely used in PDEs learning literature. In fact, is equivalent to evaluating the residuals of the equations.
For the simulation datasets (e.g., Burgers, Darcy), the ground-truth solutions are generated by high- accuracy numerical solvers, and thus satisfy the PDEs with negligible error. For the advection equation, the ground truth is given by its analytical solution. Given this, the error du = Ugt Upred reflects the deviation from the true solution. By the differential mean value theorem, we have
F(Uprea) = F(ugt) + F'(''' (Upped ugr) = F'"u"""(duu
(32)
for some u² between lgt and tpreed. Here, F is the differential operator defined by the PDE. This shows that the residual F(ugredd is proportional to the error du, with the proportionality factor being the derivative of the operator at some intermediate point. Therefore, our reported prediction errors provide an equivalent proxy for assessing whether the transferred model respects the underlying physics.
27

---

D.3 Training time of POTT
The main computational overhead of POTT lies training the transport map T, but this cost is both amortizable and reducible.
The cost of training T relatively independent of the cost of training the downstream model G. In scenarios where 9 is large or expen- sive train-as is often the case with real-world scientific models-the rel-
Amortization:
Finetuning 5min 80min
COD
DAREGRAM 11min
POTT
Darcy flow ERA5
7min
20min
100min
Table 17: Wall-clock training time comparision.
ative overhead of POTT becomes modest. We report approximate wall-clock training times Darey flow and ERA5 datasets in Tab. [7] As shown, POTT introduces moderate additional cost, which is acceptable given the substantial improvements in generalization. On simpler tasks such as Darcy, where training G is very fast, the overhead is more visible, but still within practical limits.
Reducibility: As noted in the limitations section, promising direction for reducing the training cost of POTT is to leverage large pre-trained models from related domains. For example, incorporating pre-trained multimodal foundation models, and applying parameter-efficien tuning distillation techniques to train the POTT map, can significantly reduce the overhead while potentially improving performance.
D.4
How sensitive results to the choice of metric and the accuracy of the surrogate operator? ?
Choice of metric We use the metric in Eq. (12) quantify the distributional discrepancy between Pr and ((P)). In principle, m can be any nonparametric divergence distance measure, such as the Wasserstein distance. In our implementation, we adopt the L2 metric for its simplicity and computational efficiency. Empirically, we observed that L2 provides stable gradients and effective regularization. Preliminary experiments with alternative choices (c.g., LI) yield comparable trends, suggesting robustness to the choice of metric. Nevertheless, systematically studying the impact of different metrics is an important direction for future work.
Accuracy of surrogate operator ğ: Following standard transfer learning assumptions, we rely on well-trained source model 9 that captures the physics in the source domain. is poorly trained, it may introduce misleading inductive biases. To mitigate this, we ensure that is trained with sufficient labeled data and validated carefully before transfer. practice, we also monitor the influence of the marginal-consistens consistency term during training and observe that its contribution is stable, suggesting that the learned physics from the source is indeed beneficial under the domain shift.
28


Reviews:
[{"review_id": "6xjzaUmK3k", "reviewer": "Reviewer_JnQq", "summary": "This paper introduced a transfer learning method, POTT, to solve differential equations in the context of domain shifts. The authors mainly tackle two challenges: distribution bias and operator bias. By learning an optimal transport map between the source and target domains, constrained by physics-based regularization, the method ensures both domain adaptation and perservation of physical properties. The method was evaluated on both simulation and real-world datasets.", "questions": "1. Have you analyzed the trade-off between the computational cost of the method and the potential gains in performance? Are there any comparative experiments showing how the method performs relative to existing methods in terms of computational efficiency (more interesting on a large dataset)?\n\n2. In cases where physical priors are unavailable, have you considered alternative formulations, maybe learned priors, which can better capture complex physical relationships? Did you explore this direction?\n\n3. Several typos: line 50 \"foruc\" -> \"focus\", line 268 target smaples -> target samples; Table 3: ADVERGE -> AVERAGE\n\n4. Some figures are missing color bars, for example, Figures 1, 4, 5. And can you explain how to compare those pictures across different methods?\n\n5. Is PINN a comparable method to POTT? I noticed that no PINN-based methods were included in the comparisons.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n\n1. The paper is well-written, with a clear description of the problem and method. \n2. The paper addressed an interesting and important problem in solving differential equations under domain shifts.\n3. The results demonstrate that POTT outperforms existing methods.\n\n**Weakness**\n1. For the general cases with no physical priors available, the physical regularization term $R$ relies on marginal distributions. It is unclear if this will oversimplify the underlying physics, potentially influencing performance. Additionally, the paper lacks an ablation study to evaluate how this regularization term influences the results. \n\n2. The evaluation mainly focused on synthetic datasets. Also, there is no discussion of the method's robustness to noisy or incomplete data, which are common in real-world scenarios.\n\n3. The paper mentioned that the method has a high computational cost due to its reliance on min-max optimization. This could pose challenges for scaling the method to high-dimensional DEs or more complex real-world problems.", "was_revised": true, "final_justification": "The paper presents a novel method for solving differential equations under domain shifts, and the results show that it outperforms existing methods. The reason for my score remaining as borderline accept is that the high computational cost remains a concern for practical applications, and the paper only demonstrates POTT on one real-world dataset.", "rebuttal": "Thank you for taking the time to review our paper and provide professional and constructive reviews. We are encouraged by the positive comments on the significance of the addressed problem and the paper writing. All suggestions will be carefully incorporated into final version.\n\n**_Q: Physical regularization term in general cases with no physical priors available._**\n\nThank you for this insightful question.\n\nAs discussed in **lines 191–199**, we distinguish between two scenarios regarding the availability of physical priors when constructing the physical regularization term R. When explicit physical priors are available, we leverage them directly to define R. In more general cases where no explicit priors are accessible, we instead derive R based on the marginal of the pushforward distribution.\nFirst, it is worth emphasizing that **physical priors are often more accessible than they might initially appear**. In our formulation, physical priors are not limited to the exact equations (sufficient conditions); they also include weaker but still informative necessary conditions that characterize certain properties of the system. For example, in the climate forecasting experiment, while we lack a complete physical model due to the extreme complexity of the problem, we can still rely on basic conservation principles to formulate necessary constraints. These partial insights enable the construction of R as shown in Eq. (11).\n\nSecond, in cases where no physical priors are explicitly available, POTT constructs R with marginal distributions of \\(P^r\\) . Importantly, this should not be viewed as a naive statistical constraint. As shown in Eq. (12), the regularization is guided by \\(G_s\\) learned from the source domain, which inherently captures physical properties shared across domains. This mechanism can be interpreted as **a form of learned prior**—one that encodes structural knowledge implicitly learned from the source domain. Given that source and target domains often share common physical principles, this transfer serves as an effective proxy for physical priors in the absence of explicit ones.\n\nWe acknowledge that such a learned prior may not capture all domain-specific physical nuances, particularly those unique to the target domain. However, it does retain the underlying physics shared across domains, which is often able to improve generalization.\n\nIn summary, our definition of physical priors is broad and includes both sufficient and necessary conditions, making it applicable to a wide range of problems. Even in the most challenging cases without any accessible prior knowledge, POTT provides a practical and principled approach to incorporating learned priors.\n\nWe appreciate the reviewer for raising this point—it has helped us better clarify and articulate the role and scope of the regularization term R. We will complete the discussion in Sec. 4.1\n\n**_Q: Ablation study for regularization term._**\n\nWe've presented an ablation study on the physical regularization term R in **lines 290–299** and in **Fig. 5**. As illustrated, when the physical constraint R is removed, the model degenerates into standard OTT-based transfer. In this case, the generated sample (shown in the 4th subfigure) tend to violate physical property. In contrast, incorporating the generalized physical regularization based on Eq. (12), POTT yields transformed sample (3rd subfigure) that better preserve the underlying physical structure.\n\nTo further highlight the impact of the physical regularization term, we have included additional ablation results on the same task as in Fig. 5. In this extended comparison, we isolate the effect of R by comparing models with and without it, under identical settings. The results consistently show that the absence of R leads to degraded model performance.\n\n| Finetuning | POTT w/o R | POTT |\n|------------|------------|-----------|\n| 0.0575     | 0.0543     | **0.0496** |\n\nThis provides clear empirical evidence that R is essential for enforcing physical consistency even in scenarios where explicit physical priors are unavailable.\n\n**_Q: Trade-off between the computational cost and the potential gains._**\n\nThe main computational overhead of POTT lies in training the transport map \\(T\\). However, this cost is both amortizable and reducible.\n\nAmortization: The cost of training \\(T\\) is relatively independent of the cost of training the downstream model \\(G\\). Therefore, in tasks where training \\(G\\) is computationally expensive, the relative overhead of training \\(T\\) becomes negligible. To illustrate this, we report the approximate training times on two datasets:\n\n|      \t  | Darcy | ERA5|\n|------------|------------|------|\n|Finetuning | 5min  | 80min|\n|POTT     | 20min | 100min|\n\nOn simpler tasks like Darcy, where G is small and FT is fast, POTT's additional cost is more noticeable. However, in more complex tasks like ERA5, where training G is inherently expensive, the overhead introduced by POTT becomes much less significant in proportion. In many real-world applications where labeled data is scarce or costly, this trade-off—using additional computation to gain improved generalization—can be beneficial.\n\nReducibility: As noted in the limitations section, a promising direction for reducing the training cost of POTT is to leverage large pre-trained models from related domains. For example, incorporating pre-trained multimodal foundation models, and applying parameter-efficient tuning or distillation techniques to train the POTT map, can significantly reduce the overhead while potentially improving performance.\n\n**_Q: Datasets for Evaluation._**\n\nAlthough our synthetic experiments are based on known physical equations for data generation, **they are carefully designed to reflect realistic scenarios. ** Specifically, we sample functions from diverse distributions such that each input-output pair may correspond to a different latent equation. Crucially, **no equation-specific information (e.g., symbolic expressions or coefficients) is provided during training. ** This setup mimics real-world conditions where governing equations are unknown or variable. Further details are provided in the appendix.\n\nIn addition to synthetic datasets, we also evaluate POTT on a real-world dataset—ERA5 for weather forecasting. ERA5 is based on historical climate records and naturally includes common real-world data issues. As shown in Fig. 3, POTT outperforms baseline methods under these challenging conditions.\n\n**_Q: Typos._**\n\nWe sincerely appreciate the reviewer's careful reading. All reported typos will be corrected in the final manuscript, and we will conduct a full proofread to ensure textual accuracy throughout the paper.\n\n**_Q: Color bars for Fig. 1, 4, 5_**\n\nThank you for pointing this out. \nWhile we mention in the figure captions that **brighter colors (yellow) indicate higher values**, we agree that including explicit color bars would make the visualizations clearer. **We will add them to Fig. 1, 4, and 5 in the final version of the paper.**\n\nRegarding cross-method comparison: the color mapping is consistent across Fig. 1, 4, and 5, where the color at each spatial location represents the function value—yellow indicates high values and dark blue indicates low values. The spatial distribution of colors reflects the structural properties of the function being visualized.\n\nTake Fig. 4 as an example. Col. 2 shows the ground truth, while Col. 3–5 show the predictions from POTT, FT, and COD, respectively. By visually comparing the color and shape patterns, one can observe that POTT’s predictions (Col. 3) more closely match the ground truth in both value and spatial structure, indicating better prediction accuracy and physics preservation. The same principle applies to other figures as well.\n\n**_Q: Relation with PINN-based methods._** \n\nWhile both PINN-based methods and our proposed POTT emphasize the integration of physics into the learning process, **they are not directly comparable. ** As described in **line 22–25** of the introduction, PINNs rely on the explicit form of the equations to define their physics-informed loss. Consequently, **they become inapplicable when the underlying equations are unknown or only partially specified. **\n\nIn real-world tasks, we rarely have access to fully known equations. Even when an equation is available, factors such as uncertain coefficients or perturbed functional forms can significantly impair the generalization ability of PINNs. This limitation motivates the greater adoption of data-driven methods.\n\nOur experimental setup mimics this realistic scenario. Although our synthetic experiments are based on known equations for data generation, we deliberately **do not provide any equation information (e.g., symbolic expression or parameter values) ** during training. Besides, we sample function pairs drawn from function distributions, meaning each training example may correspond to a different latent equation. Under these conditions—without explicit equation knowledge—**PINNs cannot be trained, and therefore are not suitable comparison baselines.**"}, {"review_id": "pRaTL3b218", "reviewer": "Reviewer_DNte", "summary": "The paper formulates domain shift in data-driven PDE solvers as the joint presence of distribution bias and operator bias.  To address these difficulties, the authors propose Physics-preserved Optimal Tensor Transport (POTT): an optimal-transport map between source and target product distributions that is regularised to respect available physical laws.  The push-forward of the source data through the learnt map augments the small target set, after which the pretrained source operator is fine-tuned.  A dual formulation with a Lagrange multiplier network enables end-to-end min–max optimisation.  Experiments on three simulated PDE families (Burgers, Advection, Darcy) and a cross-region ERA5 climate forecast show consistent error reductions over finetuning and recent domain-adaptation baselines.", "questions": "1. For cases without explicit priors you rely on a marginal-consistency penalty (Eq. 12).  How sensitive are results to the choice of metric *m* and to the accuracy of the surrogate operator \\( \\hat G_s \\)?  \n2.  What is the wall-clock training time of POTT relative to vanilla finetuning on the Darcy 64 × 64 task and on ERA5?  \n3. Can POTT handle ≤ 10 target samples?  Does the OT map overfit in that regime?  \n4. Have you evaluated against diffusion-based neural OT (e.g., Daniels et al., 2021) or Monge-Map networks trained without physics loss?  \n5.  Does the tensor-product OT map scale cubically with spatial dimension?  Any plans for hierarchical or low-rank variants?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths  \n1.  The distinction between distribution bias and operator bias offers a neat lens through which to view transfer learning for neural operators.  \n2.  Embedding physical constraints directly into an OT objective is intuitive and, to my knowledge, new in the context of operator transfer.  The regulariser design is flexible (explicit priors vs. marginal consistency).  \n3.   Six transfer tasks per equation plus a real-world ERA5 case demonstrate applicability to 1-D, 2-D and spatio-temporal settings.  Relative-error gains over finetuning reach 25–36 % on the hardest Darcy task.  \n4.   Removing the physics regulariser (OTT baseline) degrades qualitative fidelity, confirming its utility.\n5.   Architecture details, optimisation schedules and hardware are documented; code release is promised.\n\n##  Weaknesses  \n\n\n1. Thm 4.2 proves a consistency property of the dual saddle point but does **not** bound the transfer error nor quantify how much physics is preserved. \n2. POTT introduces two additional networks and a bi-level training loop; training cost vs. plain finetuning is not reported. \n3.  Comparisons omit recently proposed neural-OT approaches and physics-aware finetuning techniques; climate study omits domain-adaptation baselines for fairness reasons. \n4. The method is said to “diminish when only a handful of target samples are available,” but no quantitative exploration of shot regimes is given. \n5.  Several checklist items remain “\\[TODO\\]”; some typos (e.g., “foruc on” page 3).  Condensing Sections 3–4 would improve readability. \n\n---", "was_revised": true, "final_justification": "I believe the main contribution lies in the idea of \"physics preservation.\" After the rebuttal phase and reviewing the discussion between the authors and other reviewers, I gained a clearer understanding of the authors’ argument. And the  additional experiments show the \"P\" do have practical gain. I raised my score to accept.", "rebuttal": "Thank you for taking the time to review our paper and provide professional and constructive reviews. We are encouraged by the positive comments on our problem formulation, the novelty of POTT, and the thoroughness of our experimental evaluation. We will carefully incorporate all suggestions into the final version of the paper.\n\n**_Q: Thm 4.2 proves a consistency property of the dual saddle point but does not bound the transfer error nor quantify how much physics is preserved._**\n\nThank you for the insightful comment. Thm. 4.2 focuses on justifying the consistency of the dual formulation and improving the conceptual understanding of POTT. It does not provide an explicit transfer error bound nor quantify the extent to which physics is preserved. **We agree that developing such theoretical guarantees is a very important and promising direction for future work.**\n\nAs an **initial idea**, we can consider adapting the generalization bounds from semi-supervised domain adaptation. A typical result can be written as:\n$$\n\\varepsilon^t \\leq \\varepsilon^s + \\varepsilon^{lt} + \\text{dist}(\\mathcal{D}^s, \\mathcal{D}^t),\n$$\nwhich controls the target error using source error, supervised error on labeled target samples, and a distribution discrepancy term that typically relies on the feature distribution alignment.\n\nHowever, as discussed in **Sec. 3.2 (lines 150–166)**, POTT does not rely on feature alignment but instead directly corrects operator bias via target domain characterization. Thus, a coarse decomposition for POTT might take the form:\n\n$$\\varepsilon^t \\leq \\varepsilon^{lt} + \\varepsilon^{r} + \\text{dist}(\\mathcal{D}^t, \\mathcal{D}^{lt} \\cup \\mathcal{D}^{r}),$$\n\nwhere $\\varepsilon^{lt}$ and $\\varepsilon^{r}$ denote the supervised loss on labeled target data and pushforward samples respectively, and the distributional term $\\text{dist}(\\mathcal{D}^t, \\mathcal{D}^{lt} \\cup \\mathcal{D}^{r})$ reflects **how well the target domain is characterized by POTT**. Further decomposing this term, especially under the physics regularization $R(T)$, requires integrating optimal transport theory with physics-preserved learning — a challenging but promising direction we plan to explore.\n\nAs for quantifying how much physics is preserved, a simple idea is to measure this indirectly through prediction accuracy on physics-governed variables (e.g., $u$ in Darcy flow), which reflects compliance with the underlying PDE. While this provides a practical proxy, we acknowledge that developing more explicit metrics would give a more rigorous evaluation of physics preservation.\n\nWe thank the reviewer again for raising this important point. Establishing theoretical bounds on transfer error and physics preservation will be a key focus of our future work.\n\n**_Q: How sensitive are results to the choice of metric m and to the accuracy of the surrogate operator $\\hat G^s$ ?_**\n\nMetric m: We use the metric m in Eq. (12) to quantify the distributional discrepancy between $P^r_u$ and $\\hat G^s(P^r_k)$. In principle, **m can be any nonparametric divergence or distance measure, such as the Wasserstein distance.** In our implementation, we adopt the L2 metric for its simplicity and computational efficiency. Empirically, we observed that L2 provides stable gradients and effective regularization. Preliminary experiments with alternative choices (e.g., L1) yield comparable trends, suggesting robustness to the choice of metric. Nevertheless, systematically studying the impact of different metrics is an important direction for future work.\n\nAccuracy of surrogate operator $\\hat G^s$: **Following standard transfer learning assumptions, we rely on a well-trained source model $\\hat G^s$ that captures the physics in the source domain.** If $\\hat G^s$ is poorly trained, it may introduce misleading inductive biases. To mitigate this, we ensure that $\\hat G^s$ is trained with sufficient labeled data and validated carefully before transfer. In practice, we also monitor the influence of the marginal-consistency term during training and observe that its contribution is stable, suggesting that the learned physics from the source is indeed beneficial under the domain shift.\n\n**_Q: Training time of  POTT._**\n\nThe main computational overhead of POTT lies in training the transport map T, but this cost is both **amortizable** and **reducible**.\n\n**Amortization**: The cost of training T is relatively independent of the cost of training the downstream model G. In scenarios where G is large or expensive to train—as is often the case with real-world scientific models—the relative overhead of POTT becomes modest. We report approximate wall-clock training times on two representative datasets:\n|      | Finetuning | COD | DAREGRAM | POTT |\n|------|----------|------|------------|----------|\n| Darcy | 5min     | 7min | 11min | 20min |\n| ERA5 | 80min    | - | - | 100min|\n\nAs shown, POTT introduces a moderate additional cost, which is acceptable given the substantial improvements in generalization. On simpler tasks such as Darcy, where training G is very fast, the overhead is more visible, but still within practical limits.\n\n**Reducibility**: As noted in the limitations section, a promising direction for reducing the training cost of POTT is to leverage large pre-trained models from related domains. For example, incorporating pre-trained multimodal foundation models, and applying parameter-efficient tuning or distillation techniques to train the POTT map, can significantly reduce the overhead while potentially improving performance.\n\n**_Q: Comparison against diffusion-based neural OT or vanilla Monge-Map networks._**\n\nThank you for the insightful question. POTT is a general transfer learning framework and **does not impose restrictions on the underlying model used to learn the transport map T**. This flexibility means that more advanced techniques—such as diffusion-based neural OT methods—can in principle **be incorporated as backbones for learning the POTT map.** Moreover, using such methods or leveraging pre-trained diffusion-based OT models could improve both the quality and efficiency of POTT, which we highlighted as a promising direction in our **limitations section.**\n\nRegarding vanilla Monge-map networks trained without physics loss, we partially evaluated this in our paper: **in Fig. 5, we include results from standard OT-based transfer (labeled “OTT”)**, which fits transport solely via distribution alignment without any physics constraints. As discussed in **lines 215–222**, if a true Monge map exists and the network manages to learn it exactly, distribution alignment alone would suffice. However, in practice—especially under complex, high-dimensional settings—such ideal conditions rarely hold. Without physics guidance, the learned mapping might introduce significant bias or inconsistency with the target physics, leading to poor generalization. This highlights the necessity of our physics-aware training objective. Here we provide numerical results on task consistent with that in Fig. 5:\n\n|Finetuning | POTT w/o R | POTT  |\n|-------------|--------------|---------------|\n|0.0575    | 0.0543     | **0.0496** |\t\n\n**_Q: Extreme few-shot settings._**\n\nThank you for raising this important question. In extreme few-shot settings (e.g., ≤10 target samples), both finetuning and POTT indeed face significant challenges due to the risk of overfitting. **From a distribution-matching perspective, fitting a meaningful target distribution with only a few samples is fundamentally ill-posed** and prone to instability. This limitation applies broadly to all distribution-based methods.\n\nTo empirically evaluate this, we conduct a scale-down experiment on the Darcy flow task $\\mathcal{D}_2 \\to \\mathcal{D}_3$ under various numbers of labeled target samples $n^t$: \n\n|  $n^t$   |  100  |  50   |  20   |  10   |  5   |   1  |\n|----------|-------|-------|-------|-------|-------|------|\n| Tgt. Only  | 0.3493 | 0.3925 | 0.4130 | **0.4897** | **0.5107** | **0.5568** |\n| Finetuning | 0.3553 | 0.4693 | 0.4866\t | 0.5368 | 0.5521 | 0.6282 |\n| POTT     | **0.2271** | **0.3527** | **0.4126** | 0.5257 | 0.5364 | 0.6359 |\n\nWe observe that:\n\n-- As $n^t$ decreases, all methods degrade in performance.\n\n-- When $n^t >= 20$, POTT still provides advantages over both **finetuning** and **training from scratch**.\n\n-- When $n^t <= 10$, both POTT and finetuning begin to overfit, and performance starts to deteriorate significantly.\n\n-- At the extreme setting $n^t = 1$, POTT fails entirely, as the transport map becomes unreliable and may even hurt the learning of the target predictor due to **inaccurate distribution matching**.\n\nThese results clarify that POTT is most effective in the moderate few-shot regime, but may not be applicable when labeled target data is extremely scarce, which aligns with many real-world cases in scientific machine learning where acquiring a small number of high-quality labels is feasible.\n\nWe also believe that incorporating robust priors, meta-learning, or foundation models can potentially extend POTT's applicability to extreme few-shot regimes, and we consider this a meaningful direction for future work.\n\n**_Q: Computational effective variants._**\n\nThank you for the valuable question. The computational complexity does increase with spatial dimensionality. However, whether it scales cubically depends on the choice of backbone model. We acknowledge that improving the scalability of POTT is an important direction for future work. Promising approaches include using efficient backbone architectures (e.g., hierarchical structure), proper training strategy (e.g., low-rank training) or subspace decompositions. \n\n\n**_Q: Typos._**\n\nWe sincerely appreciate the reviewer's careful reading. All reported typos will be corrected in the final manuscript, and we will conduct a full proofread to ensure textual accuracy throughout the paper."}, {"review_id": "ZA0yEdsYsV", "reviewer": "Reviewer_kQhX", "summary": "The paper presents a transfer learning algorithm for operator learning problems based on optimal transport. By adding a physics-preserving regularization term to the transport problem, physical properties of the system should be preserved in the transfer. Experiments on both simulation, as well as weather/climate data show performance improvements over classical finetuning, as well as domain adaptation methods.", "questions": "- Could you provide a scaling experiment for the amount of target data needed for the method to work (better than finetuning)?\n- Could you provide an experiment with a more complicated PDE, e.g. incompressible Navier-Stokes? There it is also possible to check whether incompressibility under domain shift is retained.\n- For all experiments, could you provide baselines that are trained from scratch on target data (with the same number of samples as used for POTT)?\n- To check whether physics was preserved during the transfer procedure, could you provide e.g. the residuals of the equations?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\nThe paper presents a novel transfer learning framework based on optimal transport (POTT) and tailored to operator learning problems in physics. It is tested on both simulation, as well as experimental/real-world data.\n\n# Weaknesses\n\nThe paper suffers from numerous language issues that hinder readability -- e.g., should the title not be \"A Physics-preserving Transfer Learning Method for Differential Equations\"? There are also several typos in the manuscript. Improving the language would enhance the accessibility of the content.\n\nThe framework still needs a certain amount of samples from the target distribution to work (as also outlined in the limitations). A scaling experiment would be interesting to see.\n\nExperiments on the simulation data only consider simple equations of which only one is nonlinear. The robustness of the framework under more complicated datasets is not fully shown.\n\nWith the given experiments and baselines, it is not clear whether using POTT over training from scratch on target data is favorable.\n\nThe sub-domains used in the experiments with simulation data are nowhere defined in the main text, hindering understanding; please give a short explanation in the main text. \n\nFinally, as the title suggests, the framework should preserve physics. From the experiments it is not entirely clear to me whether this is the case. A visual analysis (as on the last page) does not seem sufficient.", "was_revised": true, "final_justification": "The paper seems a good contribution, although the results are not too strong. The main reason for my score is the language and clarity. It has to be improved in a CRV.", "rebuttal": "Thank you for taking the time to review our paper and provide professional and constructive reviews. We are encouraged by the positive comments on the novelty of POTT, and the thoroughness of our experimental evaluation. We will carefully incorporate all suggestions into the final version of the paper.\n\n**_Q: Scale experiment of the amount of target data._**\n\nThank you for raising this important question. In extreme few-shot settings (e.g., ≤10 target samples), both finetuning and POTT indeed face significant challenges due to the risk of overfitting. **From a distribution-matching perspective, fitting a meaningful target distribution with only a few samples is fundamentally ill-posed** and prone to instability. This limitation applies broadly to all distribution-based methods.\n\nTo empirically evaluate this, we conduct a scale-down experiment on the Darcy flow task $\\mathcal{D}_2 \\to \\mathcal{D}_3$ under various numbers of labeled target samples $n^t$: \n\n|  $n^t$   |  100  |  50   |  20   |  10   |  5   |   1  |\n|----------|-------|-------|-------|-------|-------|------|\n| Tgt. Only  | 0.3493 | 0.3925 | 0.4130 | **0.4897** | **0.5107** | **0.5568** |\n| Finetuning | 0.3553 | 0.4693 | 0.4866\t | 0.5368 | 0.5521 | 0.6282 |\n| POTT     | **0.2271** | **0.3527** | **0.4126** | 0.5257 | 0.5364 | 0.6359 |\n\nWe observe that:\n\n--As $n^t$ decreases, all methods degrade in performance.\n\n--When $n^t >= 20$, POTT still provides advantages over both **finetuning** and **training from scratch**.\n\n--When $n^t <= 10$, both POTT and finetuning begin to overfit, and performance starts to deteriorate significantly.\n\n--At the extreme setting $n^t = 1$, POTT fails entirely, as the transport map becomes unreliable and may even hurt the learning of the target predictor due to **inaccurate distribution matching**.\n\nThese results clarify that POTT is most effective in the moderate few-shot regime, but may not be applicable when labeled target data is extremely scarce. That said, our main focus is on scenarios where labeled data is limited but not vanishingly small, which aligns with many real-world cases in scientific machine learning where acquiring a small number of high-quality labels is feasible.\n\nWe also believe that incorporating robust priors, meta-learning, or foundation models can potentially extend POTT's applicability to extreme few-shot regimes, and we consider this a meaningful direction for future work.\n\n**_Q: Experiment with more complicated PDEs._**\n\nThank you for the insightful suggestion.\nWe acknowledge the importance of evaluating the method on more complex PDEs. While the synthetic equations in our main experiments—**Burgers’ equation and Darcy flow**—are **nonlinear and widely used** in the operator learning literature, we agree that more complicated dynamics can further demonstrate the robustness of POTT.\n\nTo this end, we provide a new experiment on the incompressible Navier–Stokes (NS) equation, a canonical example of nonlinear, multivariate, and stiff dynamics. The 2D NS equation reads: \n\n$$\n\\begin{align}\n\\partial_t w(x, t) + u(x, t) \\cdot \\nabla w(x, t) &= \\nu \\Delta w(x, t) + f(x), &\\quad x \\in (0, 1)^2, \\, t \\in (0, T] \\\\\\\\\n\\nabla \\cdot u(x, t) &= 0, &\\quad x \\in (0, 1)^2, \\, t \\in [0, T]\n\\end{align}\n$$\n\nwhere 𝑢=(𝑢,𝑣) is the velocity field, p is the pressure, and ν is the viscosity coefficient. We simulate domain shifts by varying ν∈{1e−3, 1e−4, 1e−5 }, denoted as D1, D2, D3, respectively. The settings of dataset are identical with the settings in [1]. Due to the rebuttal time constraint, we provide partial but representative results here:\n\n|    NS    | $\\mathcal{D}_1 \\to \\mathcal{D}_2$ | $\\mathcal{D}_1 \\to \\mathcal{D}_3$ |\n|$n^t$     |   50  |  100\t |  50  |  100  |\n|----------|-------|-------|------|-------|\n| Tgt. Only  | 0.2716 | 0.2368 | 0.3307 | 0.2793 |\n| Finetuning | 0.1903 | 0.1478 | 0.3188 | 0.2679 |\n| POTT     | **0.1749** | **0.1297** | **0.2706** | **0.2458** |   \n\nWe observe that in this more challenging scenario, model performance degrades significantly under low-data regimes. The target-only and finetuning baselines struggle to generalize, especially when $n^t=50$. In contrast, POTT is able to leverage the pretrained source model and capture structural information more effectively, leading to consistent improvements.\n\nThis additional experiment supports the conclusion that POTT is robust to domain shifts even in highly nonlinear and multivariate PDEs like Navier–Stokes. We will include the full experimental setup and results in the final version of the paper.\n\n**_Q: Experiments that trained from scratch on target data._**\n\nThank you for the suggestion. We agree that including target-only baselines trained from scratch helps clarify the effectiveness of POTT. Below, we provide comprehensive comparisons across all tasks.\n\n| Bur.  |  $D_1 \\to D_2$ |  $D_1 \\to D_3$ | $D_3 \\to D_2$ | \n| $n^t$ |  50   |  100  |  50  |  100  |  50  |  100  | \n|------|-------|-------|------|-------|-------|-------|\n| Tgt. only | 0.2142 | 0.1429 | 0.1173 | 0.0968 | 0.2142 | 0.1429 |\n| Finetuning | 0.2001 | 0.1191 | 0.1049 | 0.0801 | 0.1546 | 0.0938 |\n| POTT | 0.1528 | 0.0965 | 0.0950 | 0.0705 | 0.1249 | 0.0757 |\n\n| Adv.  |  $D_1 \\to D_2$ |  $D_2 \\to D_1$ | $D_3 \\to D_2$ | \n| $n^t$ |  50   |  100  |  50  |  100  |  50  |  100  | \n|------|-------|-------|------|-------|-------|-------|\n| Tgt. only  | 0.2162 | 0.1261 | 0.2585 | 0.1382 | 0.2162 | 0.1261 |\n| Finetuning | 0.0247 | 0.0143 | 0.2193 | 0.0891 | 0.1257 | 0.0723 |\n| POTT | 0.0207 | 0.0112 | 0.1872 | 0.0787 | 0.1016 | 0.0613 |\n\n| Darcy |  $D_2 \\to D_1$ |  $D_1 \\to D_3$ | $D_2 \\to D_3$ | \n| $n^t$ |  50   |  100  |  50  |  100  |  50  |  100  | \n|------|-------|-------|------|-------|-------|-------|\n| Tgt. only | 0.1815 | 0.1122 | 0.3925 | 0.2893 | 0.3925 | 0.2893 |\n| Finetuning | 0.1426 | 0.0896 | 0.1556 | 0.1605 | 0.4693 | 0.3553 |\n| POTT | 0.1362 | 0.0762 | 0.1397 | 0.1404 | 0.3527 | 0.2271 |\n\nAcross all tasks, POTT consistently outperforms both training from scratch and finetuning, demonstrating its effectiveness.\n\n**_Q: Residuals of the equations._**\n\nThank you for the suggestion. The results we report ($|| \\hat u - u||^2$, i.e., the MSE between the predicted solution $\\hat u$ and the ground truth solution $u$) are effectively **equivalent to evaluating the residuals of the equations**.\n\nFor the simulation datasets (e.g., Burgers, Darcy), the ground-truth solutions are generated by high-accuracy numerical solvers, and thus satisfy the PDEs with negligible error. For the advection equation, the ground truth is given by its analytical solution. Given this, the error **$du = u - \\hat u$** reflects the deviation from the true solution.\n\nBy the differential mean value theorem, we have \n$$F(\\hat u) = F(u) + F'(v)du = F'(v)du, $$ \nfor some $v$ between $\\hat u$ and $u$. This suggests that **when $|| \\hat u - u||^2 = (du)^2$ (the reported error) is small, the residual $F(\\hat u)$ is also small.** Therefore, our reported prediction errors provide an equivalent proxy for assessing whether the transferred model respects the underlying physics.\n\n**_Q: Typos._**\n\nThank you for pointing this out.\n\nRegarding the title: while \"Physics-preserving\" is indeed more commonly used, our usage of \"Physics-preserved\" was intentional. Specifically, we aim to emphasize that **the physics, either learned by the source model or explicitly available from the problem, are preserved during the transfer process under the guidance of the POTT map $T$.** In this sense, \"physics-preserved\" refers to the outcome of the POTT method, rather than the transfer process itself being a physics-preserving mechanism. We acknowledge that this nuance may not be immediately clear to readers, and we will consider revising the title to improve clarity and alignment with standard phrasing.\n\nAs for the rest of the manuscript, we will carefully proofread the paper to correct any remaining typos and improve the overall language quality for better readability and accessibility.\n\n**_Q: Definition of the sub-domains._**\n\nThank you for the suggestion.\n\nIn the main text, Tab. 1 provides visual examples of samples from all sub-domains used in the simulation experiments. We chose to include these visualizations because **they offer a more intuitive and accessible understanding of the domain gaps between sub-domains**, which we believe is more helpful than formal definitions for most readers.\n\nBesides, **the precise definitions of all sub-domains are provided in Appendix C.1**, and we explicitly refer to this in **the caption of Tab. 1** to ensure that interested readers can easily locate the relevant details.\n\nWe appreciate your suggestion and will consider including a brief summary of the sub-domain definitions directly in the main text or Tab. 1 in the final version to improve clarity.\n\n[1] Li, Zongyi, et al. \"Fourier neural operator for parametric partial differential equations.\" ICLR, 2021."}, {"review_id": "bA5x5vlkCt", "reviewer": "Reviewer_6yj9", "summary": "This paper studied the domain shift issue in the learning of neural operators of PDE problems, proposed a transfer learning methods which can preserve the physical relation of the equation during training.\nThis is acomplished by the optimal tensor transport method and a physical regularization.\nThree simulation bechmarks  and a real-world climate forcasting datasets were applied to validate the effectiveness of the method.", "questions": "1. The authors state that POTT is more valuable when the amount of target sample is small, yet not extremely small. However, when target sample is small, the prediction methods are too large for all the methods in the paper. When traget sample increase, simple finetuning can have good prediction on target domain. So what's the advantage of this method in practice?\n\n2. Can the authors show the computational time comparison of POTT method with other baseline methods?\n\n3. How about the physical regularization term $\\mathcal{R}(T)$ compared to the physics-informed loss like in [1]? $\\mathcal{R}(T)$  is problem-depended, making it hard to extend to arbitrary PDE problems.  On the contrary, the physics-informed loss is a universal framework once the equations are known.\n\n[1] Wang S, Wang H, Perdikaris P. Learning the solution operator of parametric partial differential equations with physics-informed DeepONets[J]. Science advances, 2021, 7(40): eabi8605.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper present a detailed analysis of existing transfer learning methods for PDE problems.\nThey  propose the Physics-preserved Optimal Tensor Transport (POTT) method to learn a physics-preserved optimal transport map between source and target domains.\nIn the case of small training samples on target domain, the POTT method can largely improve the generalization ability of the model.\nThe method is fully evaluated and analyzed.\n\nWeakness:\n\nThe Optimal Tensor Transport method is not well-presented for readers not familiar with OT theory, causing the difficulty of understanding the motivation of this work. The practical POTT loss (14) is not well-described. The physical regularization term $\\mathcal{R}(T)$ is problem-depended, without a universal framework like PINN loss, making the method hard to extend to arbitrary PDE problems.", "was_revised": true, "final_justification": "The primary reason I am unable to recommend acceptance of this paper is the limited extent of the improvement demonstrated. The chosen error metric, rMSE, is essentially the square of the more commonly used relative L2 error, which diminishes the apparent performance gains. Additionally, the manuscript contains several noticeable typographical errors.", "rebuttal": "Thank you for taking the time to review our paper and provide professional and constructive reviews. We are encouraged by the positive comments on the full evaluation and analysis. All suggestions will be carefully incorporated into final version.\n\n**_Q: Comparison between R(T) and physics-informed loss._**\n\nThank you for raising this insightful question.\n\nWe respectfully argue that the proposed physical regularization term R(T) is **more general and broadly applicable** than the physics-informed loss. Our reasoning is as follows:\n\n-- **Physics-informed loss requires fully known equations, which is a strong and often impractical assumption.** In many real-world applications, only partial physical knowledge is available (e.g., certain necessary conditions), making such methods inapplicable.\n\n-- **R(T) can leverage partial physical priors.** When partial physical knowledge (e.g., conservation properties) is available, R(T) can be constructed accordingly, e.g., **line 194**. In such cases, the physics-informed loss is unusable, whereas our method still applies. In fact, when full equation forms are available, R(T) can recover the same behavior as physics-informed loss—thus subsuming them as **a special case.**\n\n-- **R(T) also support the general settings with no explicit prior available.** As described in **line 191–199**, R(T) can be defined without any explicit physical priors by regularizing marginal distribution by the source model $G^s$.\n\nIn both our synthetic and real-world experiments, we intentionally do **not** provide the explicit equations during training or inference. Despite this, R(T) is still effective, while physics-informed losses cannot be applied.\n\nTherefore, while physics-informed losses are indeed powerful when the equations are known, we believe R(T) offers a more flexible and universal framework that is better suited to the realities of many scientific and engineering applications.\n\n**_Q: The advantage of POTT in practice._**\n\nThank you for raising this practical concern.\nWe agree that when the target domain has a large number of labeled samples, simple finetuning methods can achieve good performance. However, in many real-world applications, obtaining labeled data in the target domain is costly, time-consuming, or even infeasible—especially in scientific and industrial domains such as marine environmental forecasting, climate forecasting, and medical imaging, etc.\n\nPOTT is specifically designed for scenarios where the target sample size is limited but not extremely small. In this medium-data regime, standard finetuning may suffer from underfitting due to insufficient supervision, while methods assuming access to a large target dataset are inapplicable. POTT leverages source-domain information in a more principled and physics-aware way to improve generalization, offering a clear performance advantage in such regimes.\n\nIn short, POTT is particularly valuable **in settings where label acquisition is expensive**, in which  data is scarce, or governed by complex processes.\n\n**_Q: The computational time comparison of POTT method with other baseline methods._**\n\nThe main computational overhead of POTT lies in training the transport map $T$. However, this cost is both amortizable and reducible.\n\n**Amortization**: The cost of training $T$ is relatively independent of the cost of training the downstream model $G$. Therefore, in tasks where training $G$ is computationally expensive, the relative overhead of training $T$ becomes negligible. To illustrate this, we report the approximate training times on two datasets:\n\n|      | Finetuning | COD | DAREGRAM | POTT |\n|------|----------|------|------------|----------|\n| Darcy | 5min     | 7min | 11min | 20min |\n| ERA5 | 80min    | - | - | 100min|\n\nOn simpler tasks like Darcy, where $G$ is small and FT is fast, POTT's additional cost is more noticeable. However, in more complex tasks like ERA5, where training $G$ is inherently expensive, the overhead introduced by POTT becomes much less significant in proportion. In many real-world applications where labeled data is scarce or costly, this trade-off—using additional computation to gain improved generalization—can be beneficial.\n\n**Reducibility**: As noted in the limitations section, a promising direction for reducing the training cost of POTT is to leverage large pre-trained models from related domains. For example, incorporating pre-trained multimodal foundation models, and applying parameter-efficient tuning or distillation techniques to train the POTT map, can significantly reduce the overhead while potentially improving performance."}]

Author rebuttal:

Meta-review: This paper introduces the Physics-preserved Optimal Tensor Transport (POTT) method for transfer learning in PDE problems. The contribution is novel and relevant, addressing data-scarce scenarios where labeled target data are costly or difficult to obtain. The method shows improved generalization over finetuning, supported by extensive evaluation. In their rebuttal, the authors clarify that the proposed physical regularization term R(T) is more flexible than physics-informed losses, as it can incorporate partial priors or even operate without explicit equations. They also argue that the computational overhead is modest in large-scale tasks and could be further reduced using pre-trained models.

However, concerns remain. The exposition of Optimal Tensor Transport is not easily accessible, and the description of the POTT loss is vague. More critically, the reported absolute error levels are still too high for practical applications, raising questions about utility. Overall, this paper is borderline: it offers novelty but limited practical impact.
Decision: Accept (poster)