Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learning
Authors: Vinayak Vinayak
Ananyae Kumar bhartari
Vivek Shenoy
Venue: Accepted by TMLR ()

Abstract: Parameter estimation in inverse problems involving partial differential equations (PDEs) underpins modeling across scientific disciplines, especially when parameters vary in space or time. Physics-informed Machine Learning (PhiML) integrates PDE constraints into deep learning, but prevailing approaches depend on recursive automatic differentiation (autodiff), which produces inaccurate high-order derivatives, inflates memory usage, and underperforms in noisy settings. We propose Mollifier Layers, a lightweight, architecture-agnostic module that replaces autodiff with convolutional operations using analytically defined mollifiers. This reframing of derivative computation as smoothing integration enables efficient, noise-robust estimation of high-order derivatives directly from network outputs. Mollifier Layers attach at the output layer and require no architectural modifications. We compare them with three distinct architectures and benchmark performance across first-, second-, and fourth-order PDEs—including Langevin dynamics, heat diffusion, and reaction-diffusion systems—observing significant improvements in memory efficiency, training time and accuracy for parameter recovery across tasks. To demonstrate practical relevance, we apply Mollifier Layers to infer spatially varying epigenetic reaction rates from super-resolution chromatin imaging data—a real-world inverse problem with biomedical significance. Our results establish Mollifier Layers as an efficient and scalable tool for physics-constrained learning.

Published in Transactions on Machine Learning Research (MM/YYYY)
Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learning
Vinayak Vinayak' University of Pennsyloania
vinayako@seas wpw.ee...aeeeenneeeeeee.ee edu
Ananyac K Bhartari' University of Pernsyloania
ananyaelsscs..upen www.:eeasssssuuuucc.cc.
Vivek B Shenoy University of Pennsylnania
Cheenoy@scas.upentt edu
Reviewed on OpenReview: https: // opt.reveeeer.e..w..mmmom net/ forum? id V2F22eeee
Abstract
Parameter estimation in inverse problems involving partial differential equations (PDEs) underpins modeling across scientific disciplines, especially when parameters vary in space or time. peysics-informer Machine Learning (PhiML) integrates PDE constraints into deep learning, but prevailing approaches depend on recursive automatic differentiation (autodiff), which produces inaccurate high-order derivatives, inflates memory usage, and underperforms in noisy settings. We propose Mollifier Layers, lightweight, architecture-agnosti module that replaces autodiff with convolutional operations using analytically defined mollifiers. This reframing of derivative computation as smoothing integration enables efficient, noise- robust, estimation of high-order derivatives directly from network out puts. Mollifier Layers attach at the output layer and require no architectural modifications. We compare them with three distinct architectures and benchmark performance across first-, second-, and fourth-order PDEs includinc Langevin dynamics, heat diffusion, and reaction-diffusion destems-observee significant improvements in memory efficiency, training time and curacy for parameter recovery across tasks. To demonstrate practical relevance, apply Mollifier Layers to infer spatially varying epigenetic reaction rates from super-resolution chromatin imaging data-a a real-world inverse problem with biomedical significance. Our results establish Mollifier Layers as an efficient and scalable tool for physics-constrained learning.
1 Introduction
Parameter estimation through inverse problems constrained by PDEs arise across science and engincering, where parameters like diffusivity or conductance must be inferred from sparse, noisy measurements (Kaipio
& Somersalo (2006); Tarantola (2005): y Herrera et al. (2022) Zhang et al. (2020)). ). Traditional inference methods are computationally intensive, as they require repeatedly solving forward PDEs. These challenges grow worse in high-resolution or real-time settings, especially when governing parameters vary fine scales
data is sparse (Xun al. (203)))
Deep learning has emerged as powerful framework for such problems, with physics-informed machine learning (PhiML) approaches (Toscano et al. 22044))-ike PINNs (Raissi et al. (2019)), DeepFNO (Li et al. 2020)), and DeepONet Lu et al. (2021)) embedding PDE constraints into model training and inference. These mothods enable learning from limited observations, but rely heavily on recursive automatic differentiation (autodiff) to compute spatial and temporal derivatives (Baydin et al. (2018)). ). Autodiff based.
"These authors contributed equally.

---

Published in Transactions on Machine Learning Research (MM//YYY)
Time taken vs. Number of layers
Actual Laplacian
Predicted Laplacian
(log scale)
(log scale)
10
15
8000
8
(N0C)
6
21
2000
-5
10
10
Number of Layers
Actual Parameter vs. Predicted Parameter
Cocstraing PDE: uu A] =
N(,-()
MSE, MSE,
PhiML block
Tins Aoseiiii Communi
PINN arch/
MolliBer boyer
Parameter >
PirateNet
PINNsFormer arch
Update weights and bisses f: D(a.
-18
MSE. Σ
1|u ai{?
MSB;
Σ LÑP
Mollifying operation: 8(je) n(n m)g/mm)mm
Figure 1: Limitations of autodiff and overview of PhiML+ architecture. (a) Training time compar- ison for PINNs with and without PDE residual loss. (b) PINN-predicted vs. actual Laplacian in reaction- diffusion system. (c) PINN-predicte vs. actual forcing term in a Langevin system. (d) PhiMi+Mollifer Layer architecture replacing autodiff with mollifier-based convolution.
PhiML techniques memory-intensive and can be unstable for higher-order derivatives (Wang et al. (2024)
Fuks & Tchelepi (2020); Fourment et al. (2023); Margossian (2019) ), making such models brittle under noise and costly for deep architectures.
To overcome these limitations, we propose mollifier layers (inspired from Friedrichs 1944)); Murio et al. (1998)) lightweight; archilecturee-gnosticc modules that replace autodiff with convolution-based derivative estimation using smooth, compactly supported kernels. These layers estimate derivatives via convolution with analytic derivatives of smooth mollifier kernels, transferring differentiation off the network and implicitly regularizing the computation for stability and noise robustness.
Integrated into and benchmarked against three PhiML architectures RRissi et al. 2019h). )) PirateNet (Wang et al. (2024)), ), and PINNsFormer (Zhao et al. (2023)) mollifier layers offer a scalable, robust alterna- tive to autodiff for physics-ifformed learning. They achieve significant improvements in parameter recovery, derivative stability, and computational efficiency across diverse PDFs, including a fourth-order reaction- diffusion model (Hoo et al. (2023) where they infer the governing paramcter with high correlations while reducing the memory usage and time complexity by over an order of magnitude. Applied super-resolution microscopy (Schermelleh al. (2019)) of the human cell muclei, they enable spatially resolved inference of critical biophysical parameters from noisy data, demonstrating practical utility in real-world scientific modeling.
While our benchmarks target inverse PDEs, the underlying principle offloading high-order derivative com- putation analytic mollifier convolutions can possibly gencralize forward models (Karniadakis et al.) (2021h) (shown briefly in B.5), operator learning (Azizzadenesheli et al.) (2024)); or neural ODE systems Ruiz-Balet Balet & Zuazua (2023); Chen et al. (2018)) where gradient accuracy is crucial.
2

---

Published in Transactions on Machine Learning Research (MM/YYY)
Table 1: Peak memory usage by PINNs
Equation
MSE Loss Only
MSE + PDE Loss
Langevin (first order 1D)
0.17 GB
0.21 GB
Heat (second order 2D)
0.80 GB
1.20 GB
Reaution-Diffusion (fourth order 2D)
0.40 GB
2.70 GB
2 Motivation
As shown in prior work (Wan et al. (2024): ; Fuks & Tchelopi 2020 Fourment et al. (023): Margossian (2019)), recursive autodill despite its central role in PhiML poses key bottlenecks for learning PDEs, particularly in the accurate and efficient estimation of higher-order derivatives. To expose and quantify these, we benchmarkod an upgraded PINN (with fourier-features and residual connections, details A.1), which we will refer to as PINN on spatio-temporally varying parameter estimation. This analysis revealed three core issues: compute cost, derivative inacouracy, and memory blow-up, that directly motivated our mollifier layer design.
Motivation #1: Rising cost of high-order derivatives with model depth. Evaluating PDE residuals in PhiML, requires multiple backward passes for each order of differentiation, causing compute time to grow superlinearly with network depth (Cho et al. (2024); Suarez Cardona & Hecht (2023)). This becomes especially burdensome in complex systems that demand deeper networks. For instance, in the reaction- diffusion benchmark (system described in Sec. 4.3), training PINNs with 7 15 layers using the full loss (data + PDE) took significantly longer than using the data loss alone-an an overhead that widens with network depth (Fig la). Such costs slow down training and escalate power/hares hardware requirements, forming a major scalability barrier.
Motivation #2: Inaccuracy in higher-order derivatives for high-frequency features. Recursive autodiff is susceptible to rounding errors from repeated gradient operations. This instability becomes par- ticularly problematic when modeling spatio-temporally varying or noisy parameters. For example, a PINN trained recover the gradient a reaction-diffusion system achieved only 0.21 Laplacian correlation with ground truth (Fig 1b), indicating poor quantitative accuracy.
These issues are exacerbated with high-frequency or noisy signals, where the inherent smoothness of neural networks hinders the capture of sharp or discontinuous features (Fuks & Tchelepi (2020)). ). In a Langevin system with Gaussian white noise (Sec. 4.1.1), the PINN failed to capture variance in the inferred parameter (Fig Ic). Together, these results underscore the fragility of autodiff noisy, high-order settings, motivating the need for more stable and noisc-resilient dcrivative estimation techniques.
Computing higher-order deriva- tives through recursive autodiff requires storing intermediate gradients across layers, leading to memory usage that scales with both depth and derivative order. In experiments with paramcter cstimation for Langevin, heat, and reaction-diffusion equations using PINN(Sec. 4), adding the PDE error significantly increased GPU memory usage (hardware specs A.6) compared to training with data-only losses (Table 1). This memory blow-up limits scalability and highlights the need for more memory-efficient alternatives.
Motivation #3: High memory overhead from recursive autodiff.
3 Methods
3.1
Formulating parameter inference using PhiML
To ground our innovation, we begin with a brief overview of parameter inference for PDEs using a general PhiML franework. Without loss of generality, consider partial differential equation:
W4 + D(u, A] 0, E T
(1)
3

---

Published in Transactions on Machine Learning Research (MM/YYYY)
where 2)) is the solution of the PDE, Di:; A] is non-linear /linear ordinary partial differential operator parameterized by a(1, and the domain Q is subset in R". Subscripts denote differentiation in time or space z. Given this PDE, the aim is to infer x(t, given sparsely sampled set of u(t,z). In PhiML, the solution u(t, x) is approximated by a deep neural network uo(t, (or û), where represents the set of all trainable parameters of the network. With the autodiff formulation (Paszke el al. (2017) Raissi el al. (2019)), required gradients with respect to input variables (t, or network parameters can be computed. Additionally, X, the estimated parameter, is also an output the neural network. The actual parameter x)) is spatially or temporally varying, which can be due to patterned modulation these parameters or fluctuations due to noise. We assume that A(t, x) is unknown across Q making parameter inference challenging but broadly applicable. Combining the estimates (0, with the derivatives of á computed via autodiff, the PDE residual is defined as f = å + D(a, Å], evaluated at all points in Ω. With these tools, the optimization problem is to minimize the total mean squared error MS Frttll M.F., + M, where,
which is defined as
Data loss: MSE. = No No
Σ
la((,z') - u(tt x2)
(2)
and
1 Σ 11 PDE residual loss: MSE, = N₁ J=N
(3)
Here, N,, are the training data points available for the actual solution w(t, z) and Ny are all the points in the discretized domain for which the solution is being sought. For the initial testing of the models and characterizing their behavior, we use N₂ = NN and later validate the models on N₂ = ,. i.e., with 10% sampling. The data loss is calculated on Na, while the PDE loss on Ny.
3.2 Modified parameter inference with Mollifier Layers
Intuition behind Mollifier Layers: Recognizing the limitations of recursive autodiff, we draw from the weak- form formulation in finite element methods (Hughes) (2003)), ) where derivatives are inferred via integration against smooth test functions. This principle suggests alternative to gradient chaining: recovering deriva- lives through structured integration applied directly to the nelwork output. A similar idea underlies Sav- itzky- Golay filters (Savitzky & Golay (1964)), which compute stable numerical derivatives by fitting local polynomials within a sliding window smoothing and diffcrentiating simultancously to mitigate the effects of noise and discretization.
Mollifier layers formalize this intuition within neural architectures: rather than relying on recursive autodiff, they compute derivatives via convolution with analytic derivatives of smooth, compactly supported kernels. These mollifiers serve as localized (est functions, converting differentiation into stable smoothing operation that is mathematically grounded and memory-efficient. Compared to alternatives like Gaussian filters splines, mollifiers offer two key advantages: (1) compact support for localized and efficient computation, and (2) closed-form derivatives for stable gradient flow. Prior analysis (Murio et al. (1998)) confirms that this formulation achieves provably bounded error under discretization and measurement noise, making it well-suited for high-order and noisc-prone inverse problems.
Formal definition: Drawing inspiration from Mollifiers (Friedrichs 1944 and the weak form formulation of PDEs (details in A.7), construct modified estimator, illustrated in Fig 1d. Although the base PhiML architecture remains unchanged aside from an added mollifier layer at the output, our methodology diverges from Iraditional PhiML approach in key ways:
Rather than predicting the target field à directly, the base network outputs g. This decouples function approximation from derivative evaluation: all PDE derivatives needed for the residual are computed at the Mollifier Layer from و using fixed (non-learned) operators, avoiding recursive autodiff through the network.
4

---

Published in Transactions on Machine Learning Research (MM/YYYY)
à, is obtained at the Mollifying layer through the following operation (where is the convolution operator, y is the Mollifying function and U is the compact support of 1)):
a n = (0 * ())))))) = S 9(m)n(n- - m) dm,
(4)
At the Mollifier Layer, the higher-order derivatives w.r.t. a variable (e.g,, time or space) coordinate such as (n),,, (n), etc., are obtained through the following operations:
(n( = (0 * (m) = S ((m)n,5n m) dm,
(5)
وز3 - (0 * (;+))))n) = [ A(m)mss( - m) dm.
(6)
This formulation offloads derivative computation to analytic derivatives of a predefined mollifier 1/. Rather than using recursive autodiff, we obtain the required derivatives by convolving the learned function å with the corresponding kernel derivatives MJj Mjj. ctc.; any order-k derivative computed directly using the precomputed order-k derivative of 7/. This decoupling is the central innovation: enables efficient and stable high-orde derivatives, inherently regulorized by a low-pass filtering effect, while substantially reducing memory and compute. costs.
Implementation of Mollifier Layers. In practice the integrals in Eqs (4-6) implemented as discrete convolutions over the compact support U. The network outputs 9 the collocation grid, and the Mollifier Layer applies fixed kernels to produce = ÿ and derivative fields such as ûj = I ml and رزز = 2 * flii- Consequently, each required PDE derivative is obtained by a single convolution with the corresponding analytic kernel derivative, avoiding recursive autodiff through the network. In Fig 1d, the Mollifier Layer corresponds exactly these fixed-kernel convolutions, with kernel size determined by U. During training, we compute the PDE residual loss from à and its derivatives; gradients backpropagate through the convolution operations to update the base network parameters, while the mollifier kernels (n, My, mby) remain fixed.
The mollifier is designed to satisfy the following properties:
Smoothness (Infinitely Differentiable): The mollifier 77 is chosen from the T class to enable com- putation of all high-order PDE derivatives.
2. Compact Support: has compact support on U C R", meaning r7() = 0 for all m U. This confines integration to a finite region, reducing computational cost by restricting convolution to a fixed kernel size.
3. Non-Negativity: n(m) 2 0 for all m E U, ensuring the mollifier behaves localized averaging kernel and adheres to the following:
(a) In the limit the support of n contracts to zero, the mollifying operation converges to the identity operator, with 7/ approximating the Dirac delta 6(m).
(b) Non-negativity prevents destructive interference during integration, mitigating cancellation er- rors common in oscillatory kernels
(c) In physical systems with inherently non-negative targets (e.g., density, concentration), non- negative mollifiers preserve this constraint for consistent estimates.
We consider several mollifiers (e.g., different order polynomials and sine-based kernels) defined on appropriate compact supports; their explicit forms are provided in App A.5. An extension of mollifier layers to multi- parameter (vector-valued) inverse problems is given in App D. In our default setting we use a smooth compactly supported mollifier, e.g. the standard C bump TTx)= = cexp (exp--1-ie)) |- ||| The derivative kernels Mij -.j----- obtained by analytic differentiation of n and discretized over the same support U used
5

---

Published in Transactions on Machine Learning Research (MM/YYY)
a.
Ground truth u(t) & Predicted u(t)
b.
Learning curves for MSE loss
Learning curves for PDE loss
d.
Actual and Inferred parameter
101
onG Hoalthee and ITNO
Nae PINA
107
04-
1011
2
Ploctiin C_AnTo)
10-1
10
Acteal ACCUSIS
2000
2010
620
12000
Eportts
Expoches
Predicted vs Actual Noise (PINN)
Predicted vS Actual Noise
g. Inferred temporally varying
h. Inferred temporally varying
(Mollified PINN)
parameter (without noise)
parameter (with noise)
Artad
Actual
Bor
Postated
Eredicted Nome
Pryslooee Nolge
0.3G
9.15
0.30
9.25
2200
2013-
1.50
-1.50-
00
0.2
Actual Noise
01.4
13.6
0.8
1.0
Actual Noise
0.0 1.0
411
100 20
Time (1)
C3
40
100
Figure 2: Parameter inference for the Langevin equation using PINN and Mollified PINN. a) Ground truth and predicted u(t). b c) MSE, Data and MSE, PDE residual learning curves. d) Ground truth and inferred forcing term X(t) under Gaussian noise. c-f) Actual vs. predicted noise trends in A(t). g) Inferred time-varying X(1) without noise. h)) Inferred time-varying A(t) with added noise.
in Eqs (4-6). The kernel size is determined by the compact support U and controls the smoothing detail trade-off; sweep small set of supports and use validation performance to select U (values in Sec A.5).
With this formulation, mollifier layers provide the following key advantages over traditional autodiff-based plysics-informed approaches: (1) Computational efficiency: Derivatives are computed via a single output-layer convolution, avoiding recursive autodiff and reducing memory usage. (2) Structural de- coupling: Differentiation is shifted to fixed analytical kernels, independent of network depth. (3) Noise robustncss: As localized smoothers, mollifiers suppress high-frequency artifacts, enabling stable inference under noise.
Convergence guarantees of the Mollifier layer. [Uniform derivative consistency] Let the true field e 10) be L-Lipschitz. Noisy grid samples are 9j u(xr)) + ny at 2j jh with [nyl S Denote by (T( = (m)n(r/8) the analytic kernel from the mollifier layer and Jag 176 g. Then
Do(Jsg) - u'' S Ci8 + (h)
(7)
where Dogj- (gj+1 9j (2h)). [Proof sketch.] Combine Murio el al Murio et al. 1998) Thm 2.6 (continuous bias) with Thm 3.5 (discrete noisc); appendix A.8. Choosing ö zs Vh balances bias and variance, giving O(Th) uniform error while the backward operator norm remains O(1/8).
3.3 Inverse parameter estimation
The proposed methodology estimates the spatio-temporally varying PDE parameter Å(t,z). 2).. As discussed in Motivation#2, even though this approach yiclds estimates for smoothly varying parameters; it may un- derperform when capturing high-frequency variations due to the inherent smoothness bias of neural network outputs. To mitigate this limitation, we introduce an alternative estimation strategy applicable when A(t, x) can be expressed PDE of the form AD(u] then our final estimated PDE parameter is dyinal d. We propose that this estimation will better capture the ground truth, as á, optimized through the data loss (MSE,), reflects data variance and is better equipped capture rapidly varying spatio-temporal features. We emphasize that this is special case; the mollifier layer applies regardless of whether À is separable.
function of other predicted quantitics-in iic., when À is scparable. For example, consider

---

Published in Transactions on Machine Learning Research (MM/YYX)
4 Experiments
We evaluate our method on three PDE benchmarks of increasing complexity: 1D Langevin (first-ordr). 2D heal (second-order). and 2D reaction diffusion (fourth-order). We compare standard aacaaaddrrrriiii architectures upgraded PINNs (Raissi et al. 2019)) ) (with residual connections (Wang et al. (2021a)), spatio-temporal encodings Wang et al. (2021b]) and layer norm (Ba et al. (2016))) )) PirateNet (Wang et al. (2024)), and PINNsFormer [2223 against their mollified counterparts where applicable. These pairwise comparisons (native vs. mollfer))l)iferr)) serve as direct ablation studies, isolating the impact of mollifier layers across
Zhao et al.
models. The selected architectures reflect spectrum of design choices: PINNs as the canonical baseline, PirateNet for its stability with higher-order derivatives, and PINNsFormer for its use of attention mechanisms
(Vaswani et al. (2017): : Dosovitskiy et al. (2020); Niu et al. (2021)). Due to the high computational cost of ottention-based models, we do not include mollified PINNsFormer; mollified PINNs already achieve superior performance at a fraction of the computational cost. To further benchmark against alternatives to autodiff, we compare our method with FD-PINNS, which approximate derivatives using finite differences (Lim et al. (2022); Huang & Alkhalifah (2024); Chiu et al. (2022) ). While FD-PINNs are among the most memory- and time-efficient approaches, our results show that Mollifier Layers, by virtue of their built-in regularization, achieve superior accuracy particularly for higher-order derivatives (theoretical comparison in A.8). While all quantitative comparisons reported in the main text, plot only the Mollified-PINN variants for clarity. All supporting plots are provided in the appendix.
4.1 Oirst-Order 1D Langevin equation
To evaluate parameter inference under time-varying dynamics, we consider a simplified form of the Langevin equation-a a classical model for systems subject to both deterministic and stochastic forces. Specifically, we use the rescaled first-order ODE:
" = u + X(1),
(8)
where u(l) is the observed velocity and X() is an unknown forcing term to be inferred. This formulation preserves the essential Langevin-dynamies structure -modeing stochastic forcing transients- while sidestep- ping sccond-order complexity. As our simplest nontrivial ODE, it offers a clear benchmark for evaluating inference accuracy and robustness.
Setup. We observe u(!) at N₁ time points and infer the forcing X(1) in u, u X(1). Training minimizes a data misfit on u(t) together with the PDE residual enforced at N, collocation points; derivatives are computed via autodiff' for native models and via the Mollifier Layer for mollified variants. We report (i) correlation of the inferred forcing mean and variance against ground truth and (ii) training time and peak memory (Table 2; Fig. 2). Unless stated otherwise, we use the default mollifier y) from Sec. A.5. Under constant forcing \, all models perform similarly; we therefore focus below on settings with temporal variation and/or noise where derivative stability matters most.
4.1.1 Langevin equation with Gaussian White Noise
We simulate trajectories where the forcing is A(t) ~ NNAA, o") with A = -1 being the mean of the forcing term and a E {0, 0.44, }} being the variance capturing the noise around the mean, and we provide observations of u(t) to the models while treating A(t) as unknown. All methods optimize the standard inverse-PDF. objective consisting of data term on u(t) and a physics residual term for 4,
X(t), with mollified derivatives used for the residual in mollified variants. For consistency, we set N., N, across all experiments, using the default mollifier 77 (Sec. A.5). As shown in Fig. 2a-c for 1, both PINNs and mollified PINNs converge to the underlying function, but the latter, with our parameter estimation scheme, more accurately captures noise variations (Fig, 2d). Across noise levels, mollified PINNs consistently outperform standard PINNs in capturing these trends (Fig. 2e). The same qualitative improvements hold across architectures (Table 2).
We evaluated each mothod's vicuracy-via mean and variance correlation-ant computational effi- ciency-via training time and memory-ove five runs per noise level. Mollified variants consistently outper- formed their native counterparts in tracking temporal trends and used fewer resources on average (Table 2).
7

---

Published in Transactions on Machine Learning Research (MM//YYY)
Heat Equations
b.
Reaction-Diffusion equation
Constant parumter
Spatially varying parameler
Ground
MSE
PDE
spota Laplacion
spcon Predicted Laplacian
Predicted
Lapletin
Pradical
Figure Parameter estimation for the Heat and Reaction Diffusson equations using Mollified PINN. a) Ground truth and predictions for constant and spatially varying thermal diffusivity {(z,)) y) and Laplacians for the 2D heat equation. b) Predicted vs. actual chromatin order parameter Φs. recovered Laplacian, and training curves for the caction-diffusion system.
4.1.2 Langevin Equation with temporally varying force
We next evaluate the performance of the models in inferring noiseless, temporally varying parameter. For this, simulate trajectory with time-dependent mean forcing term, X(t) A(t), and generate simulations for various configurations of A(l) to compare the capabilities of the nalive models with their mollified versions. As shown in Fig. 2g, mollified PINNs provide very good estimate of the underlying parameter's variation. While all variants successfully capture the global mcan, native variants generally fail to capture the temporal variation the parameter.
We introduced noise (a 1) the underlying parameters by sampling X(t) ~ ((A(t), 1). As illustrated in Fig. 2h, mollified PINNs demonstrate great performance in capturing temporal trends in the forcing term compared to PINNs (Fig. 13). Similar trends are obtained for the other models (Figs. 14-16), proving that the mollifier layer lends higher accuracy predictions.
Additional results (constant forcing, other architectures, and kernel ablations) are in App. B (Secs. B.1 B.4; Figs. 5-16).
4.2 Oecond-Order 2D Heat Equation
We consider the steady-state 2D heat equation, second-order PDE modeling spatial temperature distribu- tion under thermal diffusion and external sources:
V m(x y) = 0,
where u(x,y) y) the observed temperature field, A(x,, y) the unknown thermal diffusivity, and m(x, y) is a known source term. The goal to recover {(x, y) from sparsely sampled u(x, y) and known m, a problem relevant in materials science, geophysics, and biomedical imaging for characterizing heterogeneous media.
Setup. We observe u(z, v) at N. spatial locations and infer A(x,8) y) in XV + m = 0, with m(x,y) known. Training minimizes data term the observed together with the PDE residual enforced at Ny collocation points, using autodiff for native models and the Mollifier Layer to compute 2 for mollified variants. We report correlation between inferred and ground-truth A(x, y) and computational cost (training time and peak memory; Table 2).
We evaluate all models on cases with both constant and spatially varying diffusivity. Fig 3a shows rep- resentative results for mollified PINNs, and Table summarizes correlation and efficiency across methods. Additional qualitative comparisons for other architectures provided in App. B (Figs 17-20).
8

---

Published in Transactions on Machine Learning Research (MM/YYX)
4.3 Fourth-Orde 2D Reaction-Diffusion Equation
Reaction-diffusion systems model spatial pattern formation via diffusion and local reaction kinetics, with applications in biology, chemistry, and ecology (Epstein & Xu (2016)). We study a biologically motivated system describing DNA organization in the eukaryotic/ human nucleus, where heterochromatin (inactive) and euchromatin (active) states are shaped by diffusion and epigenetic reactions such as methylation and acetylation key drivers of gene regulation (Kant et al. (2024); Vinayak et al. (2025); Dhankhar et al. (2025)). Our goal to infer the spatially varying reaction rate X(x y) from the final chromatin configuration in synthetically generated data, enabling scalable estimation of biophysical parameters. In Sec. 5, we exlend this framework to experimentally obtained super-resolution imaging data.
We adopt a phase-field model (Kant et al. (0244) with volume fractions oh. 6e, and 6n for heterochromatin, euchromatin, and nucleoplasm, respectively. v. These volume fractions satisfy the space-filling constraint Φn +
Φ. + ón 1. In Sec 4.3 we focus on the inverse problem for {(x,y) y) using the chromatin order parameter Φu = Φn - Φei on therefore omitted from subsequent expressions (it can be recovered implicitly from the constraint when needed). The dynamics of the system are deseribed as:
Døs V2pa pd 2(AØe <n),
(10)
ōt
where Pd is a chemical potential depending on Φn and de. At steady state, this yields:
V2 Ha + 2()o. - on) =.
(11)
Since Pa depends on second-order derivatives of Φs (i.e., includes terms proportional to V2 the operator V2 contains fourth-order derivatives of Oa (via (=(V")))). This makes the steady-state inverse problem effectively fourth order and a challenging testbed for stable high-order derivative learning.
Setup. We observe the final chromatin configuration (i.e., 6n y) and oe y), equivalently de (r, y)) at No spatial locations and infer the spatially varying reaction rate M(z, y) in the steady-state equation above. Training minimizes a data misfit on the obscrved chromatin fields together with the PDE residual enforced at Ny collocation points. Native models compute the required spatial derivatives via autodiff, while mollified variants compute them via the Mollifier Layer (fixed convolutions), including the Laplacian and higher-order terms arising through Vrpa. We report accuracy of X(x, y) (correlation) as well as intermediate derivative fidelity (e.g., Laplacian correlation where applicable) and computational cost (training time and peak memory; Table 2).
As shown Fig 3b, mollified PINNs accurately estimate A(x, v) and stably recover the Laplacian as a key intermediate, unlike standard PINNs, which often misestimate its magnitude (see also Fig 1). Mollified models consistently outperform their native counterparts in both accuracy and efficiency (Table 2; additional qualitative comparisons across architectures are shown in Figs 21-24).
Fig 4a further shows that mollified PINNs capture spatial noise trends in X(x, v), which the native PINNs fail to achieve, highlighting improved recovery of heterogeneous parameter fields in this high-order setting, Additional implementation details and extended results are provided in App B.3.1 and Figs 21-24.
5 Biophysical Application
DNA architecture spans multiple spatial scales, with nanoscale features such as "heterochromatin domains" lying below the diffraction limit of conventional microscopy (Ricci et al. (2015).. Super-resolutin techniques (Schermelleh et al. (2019) like STORM (Rust et al. (2006)) overcome this barrier, enabling visualization of sub-diffraction chromatin structures and their spatial heterogeneity (Fig 4b). While such imaging provides rich qualitative insight, extracting mechanistic information demands integration with physical models.
data, that mollified PINNs recover spatially heterogeneous noise trends in the inferred reaction rate field X(x,y).
Figure 4 summarizes this pipeline and its outputs. Fig 4a illustrates, on synthetic
9

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Table 2: Comparison of baseline and mollified models across PDEs and metrics. PINN: Physics-Informed Informed
Neural Network, PN: PirateNet, PF: PINNsFormer, FD: FD-PINN, +M indicates the corresponding model augmented with mollifier layers.
PDE Constraint (Order)
Metric
PINN
PINN+M
PN
PN+M
PF
FD
Time Taken (sec)
2138
1615
2414
1473
6507
1527
Mean Corr.
0.99
0.96
0.98
0.99
0.98
0.97
1D Langevin (1st)
Temporal Corr.
0.36
0.97
0.98
0.97
0.35
0.97
Peak Memory (GB)
0.21
0.16
0.77
0.11
0.33
0.14
Time Taken (soc)
2294
1582
4737
2948
10200
1478
Mean Corr.
0.81
0.99
0.13
0.22
0.73
0.82
2D Heat (2nd)
Spatial Corr.
0.21
0.99
0,16
0.20
0.04
0.76
Peak Memory (GB)
1.20
0.24
1.15
0.11
0.80
0.21
Laplacian Corr.
0.16
0.99
0.01
0.23
0.09
0.71
Time Taken (sec)
3386
335
817
125
34487
308
Mean Corr.
0.44
0.99
0.77
0.99
0.68
-0.73
2D Reaction-Diffusion (4th)
Spatial Corr.
0.17
0.84
0.25
0.91
0.47
-0.86
Peak Memory (GB)
2.75
0.23
0.48
0.12
1.90
0.21
Laplacian Corr.
0.21
0.78
0.01
0.75
0.01
-0.38
key capability needed for robust inference under measurement variability. Fig 4b shows representative STORM images (Appendix C), which serve the observational input the inverse problem. Fig 4c reports the inferred reaction-rate statistics over image regions: the predicted mean and variance of X(x, y) closely track the corresponding ground-truth statistics in controlled synthetic benchmarks and remain spatially consistent with domain-like chromatin structure in STORM-derived fields.
Together, Fig 4a-c demonstrates that mollifier layers enable stable, high-order PDE-constrained inference on imago-derived fields, supporting both accurate recovery of X(x, y) and faithful capture of spatial variability (mean/ variance) that native PINNs miss in Sec 4.3. Resolving local epigenetic reaction rates from imaging reveals the mechanistic drivers of chromatin reorganization, linking nanoscale domain remodeling to gene regulation, cancer metastasis, and cell fate memory in development and discase contexts (Vinayak et al. (0025 b Heo et al. (2023); Kant et al. (2024)). Our results thus establish mollifier layers as a robust tool for making PhiMI. scalable for integrating super-resolution imaging with biophysical modeling to extract interpretable parameters from high-dimensional biological data.
6 Discussion
By collapsing recursive higher-order automatic differentiation into a single analytic convolution, Mollifier Layers reduce both memory footprint and training time by 6-10x, as demonstrated in our experiments, while preserving robust, high-order derivative estimates. Though tested for inverse PDFs, the mollifier-based derivative layer extends naturally forward solvers (Scc. B.5), operator lcarning, and neural ODEs where stable and accurate gradients are essential. This elficiency enables scalable physics-informed machine learning to tackle complex systems like kilometer-scal. weather models (Bodnar et al. (0024); Palmer (2019) morphogenesis (Liu et al. (0024) Wyczalkowski et al. 2012)) simulations with markedly lower computational cost.
Limitations: Performance remains sensitive to the choice of mollifier kernel (see Sec. B.1.3), which must trade off noise suppression against high variance. The current hand-tuned implementation limited near boundaries and on anisotropic grids. Developing adaptive learned kernels, boundary-aware formulations, and validation strategies for adaptive meshes key directions for future work.
10

---

Published in Transactions on Machine Learning Research (MM/X (M/YYYY)
b.
Spatial noise
Microsope
Supecresolation imagee
1,6
Cul Nacks
790
199
3
PINN predicted noise
PINN-M M predicted noise
Resetion rate
tnat Denout
Scridden
Mea 14
00.99
Maa 0.00 bet ANc 12
Mar 037 Staded
Musa 2.0 Stndea
12
Artall M400 14
Sasdel-M--- 18
Figure 4: Mollified PINNs capture spatial heterogeneity in reaction-rate inference and enable PDE- constrained parameter extraction from STORM-derived chromatin fields. a) Synthetic benchmark: mollified PINNs recover spatial noise trends in the inferred reaction-rate field Mx, b) Example STORM image region preprocessed into continuous chromatin-density field (Appendix C) used as input to the inverse problem. c) Predicted mean and variance of inferred reaction rates over image regions, showing faithful recovery of reaction-rate statistics and spatial consistency with chromatin domain structure.
7 Related Works
Recent physics-informed machine learning (PhiML) approaches ranging from PINNs Raissi et al. (0199) and DeepONets (Lu et al. (2021)) to FNOs (Li et al. (020)). PINOs (Li et al. (2024b)), and PIKANs [Patra et al. (024)) have advanced PDE modeling. Our focus is on inverse problems and stable derivative computation, replacing recursive autodiff with a convolutional alternative. We review related work in these directions and outline our contributions.
Inverse Modeling: Recent work mitigates inverse-PhiMi shortcomings in complementary ways. PINNverse (Almanstötter et al. (2025)) treats parameter recovery as a constrained optimization, improving robustness to noise; H-PINN (Chandrasukmana et al. 2025)) enforces Hadamard well-posedness to regularize ill-posed tasks; Bayesian PINNs (Yang al. (2021) place probabilistic priors parameters to quantify uncertainty, albeit with significant sampling cost; and gPINNs (Yu et al. (2022)) augment the loss with gradient informa- tion to accelerate convergence; The PI-INN framework (Guan et al. (2024)) enables Bayesian inference via invertible networks but retain the full autodiff graph. Beyond the PINN family, operator-lcarning models such as the Latent Neural Operator (Wang & Wang (2024)) and Transformer-BVIP (Guo et al. (2022)) use cross-attention or boundary-aware transformers for inverse PDEs. Despite these advances, all rely on recur- sive autodiff. We replace with differentiable mollifier layer, enabling stable, memory-efficient, high-order derivatives while remaining plug-and-play.
Alternatives to Autodiff: Prior efforts to avoid recursive autodiff fall into three tracks: (i) finite-difference PhiML (Lim et al. (2022); CChiu et al. (2022); Huang & Alkhalifah (2024)) that save memory but lose high-order accuracy and suffers at high resolution, (fi) spectral PhiML (Maust et al. (2022): Patel et al.
(2022)) that gain stability yet demand bespoke bases and (iii) operator-specific networks e.g., Koopman operator nets (Lusch et al. (2018)), implicit Fourier/ Baaieer/Grrnn--iee Grocn-function networks (Li ct al. (2024a) that embed analytical derivative kernels directly in the architecture; these deliver fast inference for fixed PDE class but cannot generalize beyond the operators they hard-code. He et al. (2023) accelerate PINN training by estimat- ing PDE derivatives through Monte Carlo sampling of Caussian-smootha replacing stacked back- propagation with stochastic expectation estimators. In contrast, our approach offers a general, architect ure-agnostic alternative that enables stable, low-memory, high-order gradients even in noisy, fourth-order PDEs without sacrificing flexibility across PDE classes.
networks using Stein's identity,
Alternative inverso-PDE paradigms: Beyond alternatives to autodiff for physics-informed training, there are complementary non-neural approaches to inverse PDEs. ODIL KKrnakov et al. (2024)) optimizes discrete
11

---

Published in Transactions on Machine Learning Research (MM/YYYY)
physics/date loss over grid-valued unknown fields, whereas mollifier layers retain neural continuous surrogate and replace recursive autodiff with mollified convolutional derivative operators for stable high-order derivatives. Physics- Informedd Regression (Nielsen et al. (2025) estimates parameters via (often parameter- linear) regression/lcaste /latoo/ssaastsuureeeee structure, whereas our approach is an architecture-agnostic derivative module
that improves the efficiency and noise-robustness of high-order derivative evaluation physics-informed
neural models without assuming parameter-lincariy
8 Conclusion
Mollifier Layers offer robust, scalable alternative lo recursive autodiff, enabling stable high-order derivative estimation critical for inverse PDE learning. By decoupling differentiation from network depth, they unlock accurate, noiso-resilient inference across complex physical systems. doing so, Mollifier Layers pave the way for plysics-informed machine learning to tackle previously intractable regimes where deep models must reason stably over space, time, and noise extract scientific insight from data real-world dynamical systems.
Acknowledgments
This work was supported by NCI Award U54CA261694 (V.B.S.); NSF CEMB Grant CMMI-154857 (V.B.S.); NSF Grant DMS-2347823 (V.B.S.); NIBIB Awards R01EB017753 (V.B.S) and R01EB03086 (V.B.S.) and NIGMS Award R01GM155943 (V.B.S).
Competing Interests
The authors declare competing interests.
References
Marius Almanstötter, Roman Vetter, and Dagmar Ther. Pinnverse: Accurate parameter estimation in dif- ferential equations from noisy data with constrained physics-informed neural networks. arXiv preprint 2xXi:2554.052// 2025.
Kamyar Aviwadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, and Anima Anandkumar. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, 6(5):320-32) 2024.
Jimmy Lei Ba, Jamie Ryan Kiros, and GeofTrey E Hinton. Layer normalization. ar preprinl arXiv:1607.06/4 2016.
Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: survey. Journal of machine learning research, 18(153):1-43, 2018.
Cristian Bodnar, Wessel P Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan Weyn, Haiyu Dong, Anna Vaughan, et al. Aurora: A foundation model of the atmosphere. arXiv preprint 2rXiv:2405.1000 2024.
Susanne Brenner. The mathematical theory of finite element methods. Springer, 2008.
Yohan Chandrasukmana, Helena Margaretha, and Kie Van Ivanky Saputra. The hadamard-pinn for pde inverse problems: Convergence with distant initial guesses. Framples and Counterezamples, pp. 100175, 2025.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
12

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Pao-Hsiung Chiu, Jian Cheng Wong, Chinchun Ooi, My Ha Dao, and Yew-Soon Ong. Can-pinn: A fast differentiation method. Computer
physics-informed neural network based on coupled-auttoaati-mmmee Methods in Applied Mechanics and Engineering, 395:114909, 2022.
Junwoo Cho, Seungtae Nam, Hyummo Yang, Seok-Bae Yun, Youngjoon Hong, and Eunbyung Park. Separable physics-informed neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Monika Dhankhar, Zixian Guo, Aayush Kant, Ramin Basir, Rohit Joshi, Vinayak Vinayak, Su Chin Heo, Robert L Mauck, Melike Lakadamyali, and Vivek B Shenoy. Revealing the biophysics of lamina-associated domain formation by integrating theoretical modeling and high-resolutton imaging. Nature Communicaa- tions, 16(1)9 2025.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa. Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXic:2010. 11929, 2020.
Irving R Epstein and Bing Xu. Reaction-diis diffusion processes at the nano-and microscales. Nature nonotech- nology, 11 144333122319, 2016. ISSN 1748-3387.
Bengt Fornberg. Generation of finite difference formulas arbitrarily spaced grids. Mathematics of com- pulation, 51(184):699 706, 1988.
Mathieu Fourment, Christiaan Swanepoel, Jared G Galloway, Xiang Ji, Karthik Gangavarapu, Mare A Suchard, and Frederick A Matsen Iv. Automatic differentiation is no panacea for phylogenetic gradient computation. Genome biology and evolution, 156))eeva,099 2023.
O. Friedrichs. The identity of weak and strong extensions of differential operators. Transactions of the American Mathematical Sociely, 55(1):132-115 151, 1944. ISSN 00029947, 10886850. doi: 10. 10.2307/1990143, URL http://weet http://ww..tttooooooooooo wastor.orr//taaee/199000//.
Olga Fuks and Hamdi Tchelepi. Limitations of physics informed machine learning for nonlinear two-phase transport in porous media. Journal of Machine Learning for Modeling and Computing, 1(1), 2020. ISSN 2689-3967.
Nick Gilbert, Susan Gilchrist, and Wendy A Bickmore. Chromatin organization in the mammalian nucleus. International review of cylology, 242:283 336, 2005.
Xiaofei Guan, Xintong Wang, Hao Wu, Zihao Yang, and Peng Yu. Efficient bayesian inference using physics- informed invertible neural networks for inverse problems. Machine Learning: Science and Technology, 5 (3):035026, 2024.
Ruchi Guo, Shuhao Cao, and Long Chen. Transformer meets boundary value inverse problems. preprint arXio:2209.1499 14977, 2022.
ar
Di He, Shanda Li, Wenlei Shi, Xiaotian Gao, Jia Zhang, Jiang Bian, Liwei Wang, and Tie- Yan Liu. Learn- ing physics-informed neural networks without stacked pack-propagation. In International conference on artificial intelligence and statistics, pp. 3034-3047. PMLR, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
S. J. Heo, S. Thakur, X. Chen, C. Loebel, B. Xia, R. McBeath, J. A. Burdick, V. B. Shenoy, R. L. Mauck, and M. Lakadamyali. Aberrant chromatin reorganization in cells from discased fibrous connective tissue response to altered chemomechanical cues. Nat Biomed Eng, 722):177-191, 2023. ISSN 2157-846X (Electronic) 2157-846X (Linking). doi: 10.0038/441551--22--001333 URL https: sttps:///ww.nnt www.wwwwww.c.www..w...ww...w...w nlm nih. gov/pubbmed/35599602
Paulo Herrera, Miguel Angel Marazuela, and Thilo Hofmann. Parameter estimation and uncertainty analysis in hydrological modeling. Wiley Interdisciplinary Reviews: Water, 9(1):e1569, 2022. ISSN 2049- 1948.
13

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Xinquan Huang and Tariq Alkhalifah. Efficient physics-informed neural networks using hash encoding. Journal of compulational physics, 501:112760, 2024.
Thomas JR Hughes. The finite element method: linear static and dynamic finite element analysis. Courier Corporation, 2003.
Jari Kaipio and Erkki Somersalo. Statistical and computational inverse problems, volume 160. Springer Science & Business Media, 2006.
A. Kant, Z. Guo, V. Vinayak, M. V. Neguembor, W. S. Li, V. Agrawal, E. Pujadas, L. Almassalha, V. Backman, M. Lakadamyali, M. P. Cosma, and V. B. Shenoy. Active transcription and epige- netic reactions synergistically regulate meso-scale genomic organization. Nat Commun, 15(1):4338 2024. ISSN 2041-1723 (Electronic) 2041-1723 (Linking). doi: 10.0088/41144---22--6008-- URL
https: bttps::/www...r http:/www..cbbbbbctcc www.inlm.....m......mmmmmm.....m .nw. www..i.......iioe htth.gov/puumee//88711122
Petr Karnakov, Sergey Litvinov, and Petros Koumoutsakos, Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks. PNAS nerus, (11::gaac005, 2024.
George Em Karniadakis, Ioamnis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics- informed machine learning. Nature Reviews Physics, 3(6): 440, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412 6980, 2014.
Sanjiva K Lole. Compact finite diffcrence schomes with spectral-like resolution. Journal of computational plysiccs 10311):1 42, 1992.
Shengyan Li, Qi Sun, Xuejun Xu, and Bowen Zheng. Neural green's function accelerated iterative methods for solving indefinite boundary value problems. orXiv preprint arXiv:24100009 2024a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprinl ar Xix::22010. 088855, 2020.
Zongyi Li, Hongkai Zheng, Nikola Kovachki, David Jin, Haoxuan Chen, Burigede Liu, Kamyar Azizzade- nesheli, and Anima Anandkumar. Plysics-informed neural operator for learning partial differential equa- tions. ACM/JMS Journal of Data Science, 1(3):1-27, 2024b.
Kart Leong Lim, Rahul Dutta, and Mihai Rotaru. Physics informed neural network using finite difference method. In 2022 IEEE International Conference Systems, Man, and Cybernetics (SMC), pp. 1828- 1833. IEEE, 2022.
Yue Liu, Xufeng Xue, Shiyu Sun, Norio Kobayashi, Yung Su Kim, and Jianping Fu. Morphogenesis beyond in vivo. Nature Reviews Physics, 6(1):28-44 044:28844444 2024.
Nya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXio preprinl arXiv:1608.0300 2016.
Lu Lu, Pengzhan Jin, Guofei Pang. Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via. deeponet based the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218 229, 2021. ISSN 2522-5839. doi: 10. 10.1033354 URL httpss///:// org/ 10. 100.1003//22222 021-22256-221-03383
Bethany Lusch, J Nathan Kutz, and Steven L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communicationns, 9(1):4950, 2018
Charles C Margossian. review of automatic differentiation and its efficient implementation. Wiley inter- disciplinary reviews: data mining and knowledge discovery, 94) 1305, 2019.
14

---

Published in Transactions on Machine Learning Research (MM/YYY)
Haydn Maust, Zongyi Li, Yixuan Wang, Daniel Leibovici, Oscar Bruno, Thomas Hou, and Anima Anand- kumar. Fourier continuation for exact derivative computation in physics-informed neural operators. arXiv preprint XrXi::2211. 159600 2022.
DA Murio, Carlos Mejía, and S Zhan. Discrete mollification and automatic numerical differentiation. Computers 8 Mathematics with Applications, 35(5):1-16, 1998.
Elizbar A Nadaraya. On estimating regression. Theory of Probubility & Its Applications, 9(1):141 142, 1964.
Jonas Søcborg Nielsen, Marcus Galea Jacobsen, Albert Brincker Olson, Mads Peter Sorensen, and Allan Peter Engsig-Karup. thysics-informae regression: Parameter estimation in parameter- linear nonlinear dynamic models. arXiv preprint Xiv:22508. 19249, 2025.
Zhaoyang Niu, Guoqiang Zhong, and Hui Yu. A review on the attention mechanism of deep learning. Newrocomputing, 452:48-62, 2021.
TN Palmer. Stochastic weather and climate modcls. Nature Reviews Physics, 1(7):463-471, 2019.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Ravi G Patel, Indu Manickam, Nathaniel A Trask, Mitchell A Wood, Myoungkyu Lee, Ignacio Tomas, and Eric C Cyr. Thermodynamically consistent onysiee-ifformed neural networks for hyperbolic systems. Journal of Computational Physics, 449:110754, 2022.
Subhajit Patra, Sonali Panda, Bikram Keshari Parida, Mahima Arya, Kurt Jacobs, Denys Bondar, and Abhijit Sen. Physics informed kolmogorov-arnold neural networks for dynamical analysis via efficent-kan and wav-kan. Xixiv preprint. arXiv:2407.1883 183733 2024.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informet neural networks: deep learn- ing framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019. ISSN 0021-9991.
Maria Aurelia Ricci, Carlo Manzo, María Filomena Carcía-Parajo, Melike Lakadamyali, and Maria Pia Cosma. Chromatin fibers are formed by heterogeneous groups of mucleosomes in vivo. Cell, 160(6): 1145-1158, 2015.
Domence Ruiz-Balet and Enrique Zuazua. Neural ode control for classification, approximation, and transport. SIAM Review, 65(3))735 773, 2023.
Michael J Rust, Mark Bates, and Xiaowei Zhuang. Stochastic optical reconstruction microscopy (storm) provides sub-diffraction-lim limit image resolution. Nature methods, 3(10):793, 2006.
Abraham Savitzky and Marcel JE Golay. Smoothing and differentiation of data by simplified least squares procedures. Analytical chemistry, (6(8):1627-163)) 1964.
Lothar Schermelleh, Alexia Ferrand, Thomas Huser, Christian Eggeling, Markus Sauer, Oliver Biehlmaier, and Gregor PC Drummen. Super-rosolution microscopy demystified. Nature cell biology, (1())72884 2019.
-as--en Suarez Cardona and Michael Hecht. Polynomial differentiation decreases the training time complexity of physics-informed neural networks and strengthens their approximation power. Machine Learning: Science and Technology, 4, 2023.
Albert Tarantola. Inverse problem theory and methods for model parameter estimation. SIAM, 2005.
Juan Diego Toscano, Vivek Oommen, Alan John Varghese, Zongren Zou, Nazanin Ahmadi Daryakenari, Chenxi Wu, and George Em Karniadakis. From pinns to pikans: Recent advances in physics- informed machine learning. Xiu preprint ar Xrx:v:2410 10. 13228, 2024.
15

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informalion processing systems, 30, 2017.
Vinayak Vinayak, Ramin Basir, Rosela. Golloshi, Joshua Toth, Lucas Sant' Anna, Melike Lakadamyali, Rachel Patton McCord, and Vivek B Shenoy. Polymer model integrates imaging and sequencing to reveal how nanoscale heterochromatin domains influence gene expression. Nature Communications, 16(1):3816, 2025.
Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal Scientific Computing, 4365):13055-3301, 2021a. ISSN 1064-8275.
Sifan Wang. Hanwen Wang, and Paris Perdikaris. On the cigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks. Compuler Methods in Applied Mechanics and Engineering, 384:113938, 2021b.
Sifan Wang, Bowen Li, Yuhan Chen, and Paris Perdikaris. Piratenets: Physics-informed deep learning with residual adaptive networks. arXi preprint or xrXiv22402000996 2024.
Tian Wang and Chuang Wang. Latent neural operator for solving forward and inverse pde problems. arXiiv preprint Xiv:2440 03923, 2024.
Geoffrey S Watson. Smooth regression analysis. Sankhya: The Indian Journal of Statistics, Series A, pp. 359 372, 1964.
Matthew Wyczalkowski, Zi Chen, Benjamen A Filas, Victor Varner, and Larry A Taber. Computational models for mechanics of morphogenesis. Birth defects research port C: Embryo today: Reviews, 96(2):132- 152, 2012.
Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. Advances in neural information processing systems, 32, 2019.
X. Xun, J. Cao, B. Mallick, R. J. Carroll, and A. Maity. Parameter estimation of partial differential equation models. J Am Stat Assoc, 108(503), 2013. ISSN 0162-1459 (Print) 1537-274X (Electronic) 0162- 1459 (Linking). doi: 00080//06221459.211...88800 24363476
URL https://www.ncr ncbi. nlm. nih, gov/pubmed/
Liu Yang, Xuhui Meng, and George Em Karniadakis. B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
Jeremy Yu, Lu Lu, Xuhni Meng, and George Em Karniadakis. Gradient-enhaneed physics-informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering, 393:114823, 2022.
Shaoqing Zhang, Zhengyu Liu, Xuefeng Zhang, Xinrong Wa, Guijun Han, Yuxin Zhao, Xiaolin Yu, Chang Liu, Yun Liu, and Shu Wu. Coupled data assimilation and parameter estimation in coupled ocean atmosphere models: review. Climate Dynamics, 54:5127 5144, 2020. ISSN 0930-7575.
Zhiyuan Zhao, Xueying Ding, and B Aditya Prakash. Pinnsformer: transformer-based framework for physics-informed neural networks. XiXiv preprint arXic:2307. 11833, 2023.
Olgierd Cecil Zienkiewicz, Robert Leroy Taylor, and Jian Z Zhu. The finite element method: its basis and fundamentals. Elsevier, 2005.
16

---

Published in Transactions on Machine Learning Research (MM/YYY)
A Details of the models
A.1 Details of the PINNs implementation
Significant progress has been made in improving PINNs since they first came out (Raissi et al. (2019) specially showing that gradient flow can be significantly improved by adding residual connections (Wang et al. (2021a)). Additionally, it is well known that layer normalization helps with gradient smoothening and prevents explosion (Xu al. (2019)). Since our aim to accurately capture gradients and account for this progress made, used upgraded PINN, which has the following two features in addition the base PINN architecture:
Residual Connections: Also known as skip connections, residual connections (He et al. (2016)) help mitigate the vanishing gradient problem and facilitate the training of deep neural networks. They can be incorporated into any layer by modifying the standard transformation in a multi-layer perceptron (MLP). Given an input a node in linear layer L, the standard output f(x) is defined as:
( (W b)
(12)
where Φ, W, and b represent the activation function, weights, and biases, respectively. To introduce residual (skip) connection between layer and layer 1, the modified output (to(()) is given by:
fte (x) d(Wx x + b) +
(13)
This modification allows gradients to propagate more effectively through deeper layers, improving both convergence and model performance.
2. Layer Normalization: Layer normalization (Ba al. (2016) stabilizes and accolerates the training of deep neural networks by normalizing activations across features within each training example, rather than across the batch dimension as in batch normalization. This ensures consistent normal- ization even for small batch sizes, making particularly useful in recurrent and transformer-based architectures. Given input features I, the output after applying layer normalization is:
((x) Efx] E[2]
7 +B Var(x)+ +c
(14)
where [[[x] and Var(z) are the mean and variance computed over the feature dimension of each input, and 7 and 8 are learnable scaling and shifting parameters. This transformation ensures stable feature distributions, improving training convergence and model performance.
A.2 Details of the PirateNet implementation
We adopt the original architectures with their components intact and fine-tune each one by varying their depth specifically for our problems, as detailed in Scction B.
A.3 Details of the PINNsFormer implementation
We adopt the original architectures with their core components intact and fine-tune each one by varying their depth specifically for our problems, as detailed in Section B.
A.4
Common training choices across models
We use the following choices for all the models:
Fourier Features: As shown through previous works (Wang al. (0211)), input features mapped to higher-dimensional space using Fourier features perform better in capturing high frequency
17

---

Published in Transactions on Machine Learning Research (MM/YYYY)
signals in the output. Given an imput x, define set of frequencies w, within interval I R. Specifically, we set 7 = [-3,3] 3] and select w, at equal intervals within this range. The Fourier feature transformation then maps each input a as follows:
- (oo((w))),,co(((1)))) ... (wn(w,))) cos(a))))
(15)
where sin(x) and cos(2) are applied component-wise each feature in x. This transformation expands the feature space, allowing the model capture periodic patterns and high-frequency variations in the data. By incorporating both sine and cosine terms, Fourier features provide richer oscilla- tory representation, improving the model's ability to approximate functions with diverse frequency components. This enhanced representation strengthens the model's 's ability to learn and generalize frequency-dependent behaviors.
2. Adam Optimization: Adam is an adaptive optimization algorithm (Kingma & Ba. (014)) widely used for training deep learning models. It combines the advantages of Momentum and RMSprrp, adapting the learning rate for each parameter based on both the first moment (mean of gradients) and the second moment (uncentered variance gradients). This adaptive mechanism allows Adam to efficiently navigate complex optimization landscapes, cnsuring faster convergence, especially problems involving sparse gradients or noisy datasets.
Adam maintains two moving averages for each parameter A:
my: First moment (mean of gradients).
Second moment (uncentered variance of gradients).
At time step i, these moments are updated follows:
m, = --m4-1 + (1 B1)g.
(16)
= 120 -1- -B2)g
(17)
where:
9e the gradient of the loss function with respect 8 at time step t.
31 and B2 the exponential decay rates for the first and second moments, typically set to
31 = 0.9 and $2 0.999.
To correct the bias toward zero in the initial steps, Adam applies bias correction:
m
12 3
m.
(18)
B
Finally, the parameters updated using:
0, 01-11
mi
(19)
Vbg +
where:
is the learning rate.
is small constant (typically 10-8) to prevent division by zero.
Adam's adaptive learning rates make effective across a wide range problems, particularly those involving noisy or sparse gradients.
18

---

Published in Transactions on Machine Learning Research (MM/YYYY)
3. Cosine Learning Rate Decay [Loshchilov & Hutter (0166) ): In deep learning, the learning rate is often initialized at relatively high value and progressively reduced to refine the model and prevent overshooting minima. A common approach for this adjustment is cosine anneating, which schedules the learning rate to follow a cosine curve, ensuring a smooth and gradual decay. Compared to step decay, cosine decay has been shown to improve convergence and generalization.
The learning rate at time step t is defined as:
17.
if f To
0t
c
17 cos
L- 7-ho
2
(20)
where:
y is the initial learning rate.
is the minimum learning rate (lypically close to zero).
T is the total number of training iterations or epochs.
is the current iteration or epoch.
To the point at which the learning rate transitions from constant. to decreasing.
This docay strategy cnsures smooth reduction in the learning rate, allowing the model gradually settle into well-generalized region of the loss landscape. When combined with Adam oplimization, cosine annealing lead to faster convergence and improved performance unseen data.
A.5 Choice of Mollifying function
With the constraints of smoothness, compact support, and non-negativity outlined in the main text, we define three mollifying functions explore the behavior of our methodology. We consider a second-order polynomial, a fourth-order polynomial, and sine function as potential mollification functions, along with the original exponential mollifier kernel. These functions are defined follows:
Second-order polynomial:
fa(x) = -(x² - R22)
(21)
Fourth-order polynomial:
f4(x) =1xx² -12122
(22)
Sine function:
( n Rı: n
fxi =sin
(23)
2 2
Original mollification function:
(x) = 1-22
(24)
where (R) with Bx( representing the L2-norm ball centered at zero. We refer to Bx(0) as U in the main text for simplicity.
The kernel size is defined relative to the grid resolution, with values set as follows:
Langevin equation: 0.01
Heat equation: 0.005
Reaction-dlifusion system: 0.05
Thus, kernel size of corresponds to:
19

---

Published in Transactions on Machine Learning Research (MM/YYN)
R = 0.3 for the Langevin equation,
R 0.015 for the heat equation,
R = 0.15 for the reaction-diffusion system.
We benchmarked the performance of these kernels using the Langevin equation with a constant porameter with noise, with details provided in Section B.1.
A.6 Hardware specifications
Our computing core consists of nodes equipped with four Tesla K80 GPU cards, each containing two GK210GL GPUs, resulting in eight GPUs per node. The GK210GL GPU architecture features 13 Streaming Multiprocessors (SMs), with each SM comprising 192 CUDA cores, yiclding a total of 2,496 CUDA cores per GPU. For each training run, we utilize a single node.
A.7 Weak Form Motivation
The success of the Finite Element Method (FEM), which employs weak form approach solve Partial Differential Equations (PDEs) (Brenner (2008 Zienkiewicz et al. (2005)), has inspired the development of the mollification operation (Friedrichs (1944) presented in this work. Lets begin with function v, u, : U + R. here u is the actual solution to PDE, is the source term and v is the weak form approximation of u". the weak form PDE, we aim to minimize the residual, which can be expressed as:
Jo v(v))()) S f(y)d()) dy =0 C (U)
(25)
We say w" in the weak sense if,
[ v(w)(()) -
= -/ a'(v)d'()) "(())(())
dy Va (U)
(26)
This formulation helps us avoid the direct computation of higher-order derivatives, which would otherwise complicate the process. By approximating the PDE residual with derivatives, we can improve the accuracy of the solution. This leads us to propose an approach in which all derivatives are approximated in the weak form.
To achieve this, employ well-cstablished approximation of functions in Sobolev spaces using mollification.
This allows us to express the PDE residual in the weak form as:
1/o n n/ ((((=)))()))))) dy { (.(o-v)((0)) v)f(v) dy
v)v(v)
(27)
Here, Dd in the weak sense and (t) is compact support bounded function in (-c,c). Here n, also known the mollifying function, is non-negative, infinitely differentiable and is defined as:
na(t) = nn())
(28)
((x) = e 11
V||0||2 < 1
(29)
7(x) = A||z||| 2
(30)
We further relax the above residual by doing the following:
20

---

Published in Transactions on Machine Learning Research (MM/YYY)
V (2(xx)))(()) ne(x
f(v) (oc(x--)) v) dy 1 ne(o - -1) dy f(0)
(31)
+ [(x) - 1( nelo - v) ((v) dy
Note that the residual part our importance is:
/ v)u(() - f(0))
(32)
The other term tends to zero decrease
/ ne(x nn((x-v) v) v) f(y) dy
= =1ʃ ne(x (o(x=-)(f(()) f(1)) dy
((x)
(33)
I
(= [f(x) - f(y) dy.
S e Bla.s)
1
" (-- [1xx) f(v) dy S -
et SmLcs (f(z)
(34)
6" B(z,c)
where C = SuPteR" m1(). Using the Lebesgue Differentiation Theorem, we can state that as - 0:
I B(r,c)
1 - f(y)] dy ~
(35)
Now, we can see that u is our weak derivative (approximated by a neural network). Thus, the expression becomes:
Upred = 7) NN8
(36)
Here NNg is the final output form our Neural network layers.
As we are working on the grid (G), so express the PDE residual with the help of Mollifying operations as:
EEEG Dine NNg(#)-J(a))
(37)
GG
Here x sampled randomly from a predefined grid of points in the domain.
A.8 Convergence Analysis of the Mollifier Layer
Let S: [0. R be continuously differentiable and L-Lipschitz. Denote the uniform grid spacing by h > 0 and the kernel half-width by 8 > 0. For a nonnegative bump p E C~ ([) with unit mass, define the mollifier
(Jaf)(z) 1 po(r))() r)dr,
po(r) 8-1 8 p(r/8).
(38)
Measured data are 9, f(1,) n I. = jh with Ing1 S E. Throughout, C, C1, Cz.... denote positive constants independent of 5, h, and
21

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Mean-square consistency.
Murio et al. (9988) ) For f (2,, 1)),
||ss - f1l + 0 as 6 + 0.
(39)
Uniform consistency (Murio et al. 1998 )
Js flloo I CLÓ,
|1 - ff & CLS.
(40)
Discrete mollification (Murio al. (1998))
|sg - A1 S c((++++s).
(41)
Finite-difference accuracy (Murio et al. 1998
Do(Jsg) + J.Jof(r)) & Ch,,
Dogj
9j+1 9j-1
(42)
2h
Uniform derivative consistency
MMurio et al. 1998
[Do(Ja) f' C(6 + 1² +
(43)
Choosing = /h and A²2 E minimises the bound, giving O( uniform error while the backward operator norm ||] remains (1188).
For higher-order derivatives, differentiating under the integral sign yields
(J=
(44)
where the superscript acts on the kernel. The same arguments with a controd stencil of longth 2n give
(Js() || C, + h
(45)
Finally, in the case n and absorbing h²2 into the constants we obtain
[a(Jsf) fl < C2)
(46)
which is the bound quoted the main text.
Comparison with finiteddifeerene (FD) derivatives. To clarify the distinction between finite- difference (FD) schemes and mollifier layers, summarize both the theoretical error behavior and the empirical outcomes observed in our experiments.
Finile differences. FD schemes approximate derivatives by forming weighted sums of nearby grid values. For higher-order derivatives, the associated coefficients become increasingly oscillatory and extreme (see Fornberg (1988); Lele (1992)). which amplifies any measurement noise. Analytically, for derivative order m on a grid with spacing h, the FD error bound includes noise-amplificttion term of order e/h"M in addition to truncation error terms (hh) depending on the scheme order p. Thus, m grows, noise is magnified by the factor 1/h"", rendering high-order FD estimates unstable.
Mollifier layers. In contrast, mollifier-basee differentiation first convolves the signal with smooth, com- pactly supported kernel before differentiation. This convolution damps high-frequency components, prevent- ing noise amplification. As derived, the derivative crror bound for mollifiers takes the form
[Dag)))) ul ||x < C + C2 + e),
Importantly, the growth with derivative order is absorbed into the mollifier kernel, so stability preserved even for high-order PDEs.
Empirical validation. Our experiments confirm this theoretical distinction:
22

---

Published in Transactions on Machine Learning Research (MM/YYY)
For the 1D Langevin equation (first order). FD-PINNs and mollifie--PINNS perform comparably.
For the 2D heat equation (second order), FD-PINN accuracy deteriorates, while mollifier-PINNs
remain stable.
For the 2D reaction-diffusion equation (fourth order), FD-PINNs collapse entirely, whereas mollifier-
PINNs retain high accuracy.
Together, both the analytic bounds and experimental results demonstrate that FD-based differentiation suffers from noise amplification that scales unfavorably with derivative order, while mollifier layers- by acting as built-in low-pass filter-maintale stable, accurate performance in high-order and noisy settings.
B Application-wise details
B.1
Langevin equation
B.1.1 Training Setup for All Models
PINN architechures: The PINN architecture for prodicting å consists of seven layers, cach with 1000 nodes and ReLU activation functions. To enhance temporal encoding, augment the input time t with Fourier features (details in appendix A). The model is trained using the Adam optimizer with a cosine annealing learning rate schedule. The corresponding mollified PINN maintains the same architecture and training epochs but inchudes the mollifying layer.
PirateNet arch: PirateNct models the stochastic ordinary differential equation 2, u + X(t) with an eight- layer fully connected residual network, each layer having 256 hidden units and tanh activations. The scalar time imput t is first lifted into 256-dimensional Fourier feature embedding (scale factor 2) to capture high-frequency temporal structure. Each layer applies custom PI Modified Bottleneck block: three-layer MLP whose residual update is mixed via two learnable nonlincar functions u(-) and v(-), and whose skip connection is scaled by a trainable parameter a. Finally, two parallel output heads prediet the solution @()) and the auxiliary forcing cocfficient Å(t), enabling the network to learn both the state evolution and the driving noise in one joint framework.
PINNsFormer arch: When applied to the Langevin equation u, = u + A(t): PINNsformer configured as a four-layer transformer -ncoder-docoder with model dimension 64 and four attention heads per layer. The input time t is first encoded using learned sinusoidal embeddings. Each transformer layer applies multi- head self-attention followed by a three-layer feed-forward block activated by WaveAct, learnable mixture of sin(x) and cos(x). Full residual connections and layer normalization are maintained throughout. Two decoder heads then output the state â(t) and the auxiliary estimate A((), allowing the attention-based model to jointly learn deterministic evolution and stochastic forcing.
All models are trained on a grid sampled from the interval [0, 1], using batch size of 200 for 1000 epochs. The initial learning rate is set to 10-3 with cosine decay scheduling. The Adam optimizer is configured with parameters 3 = 0.9, B2 0.999.
B.1.2 Evaluation on the Langevin Equation
To lest the models, we first. infer a constant forcing term for the Langevin equation. As shown in Fig. 5-7, all models successfully predict the constant forcing term. We then extend our analysis to Gaussian white noise predictions, with results summarized in Section 4.1 and Table 2. The figures for all the models are Fig. 8-10. Mollified versions outperform the native nets in all cases.
Thereafter, we also test the models temporally varying signal along with its noisy version Fig. 13-16. Here too the mollified models outperform the native models.
B.1.3 Impact of Functional Choices
The effectiveness of Mollifier-Nets depends two key parameters:
23

---

Published in Transactions on Machine Learning Research (MM/YYX)
1. The order of the mollifying function 77.
The kernel size, Le., the integration domain U.
For the Langevin equation with constant noisy parameter, various configurations accurately estimate the mean (Fig. 11), but they differ in their ability to capture noise variations.
Kernel Size Effect: As shown in Fig. 11, for second-order polynomial mollifier, increasing the kernel size degrades performance on large noise, while a small kernel size fails to capture finer noise variations. An optimal kernel size of 10-15 balances noise capture effectively. This behavior can be altributed to the frequency distribution of the underlying function.
Mollifier Order Effect: As shown in Fig. 12, higher-order polynomials perform worse at capturing noise compared to lower-order mollifying functions.
Conclusively, both polynomial order and kernel size can be optimized. While low-order polynomial with a moderate kernel size is ideal for extracting Gaussian white noise in the Langevin setting, these choices may vary for different learnable functions.
B.2 Heat equation
TRAINING SETUP
The neural network consists of 10 layers, each with 250 nodes and tanh activation functions. Spatial Fourier features are used for positional encoding. Training is performed using the Adam optimizer with learning rate that progressively decreases following cosine decay schedule. The mollified version consists of the mollifier layer in addition to the base arch with original mollifying function.
PINN architecture:
PirateNet Trchitecturc:To solve the two-dimensional heat equation, PirateNet uses the same eight residual layers with 256 units each and tanh nonlinearities. The spatial coordinates (x,,)) are first transformed via a 256-dimensional Fourier embedding (scale 2) to enrich spatial frequencies. Inside each PI Modified Bottleneck block, the network mixes the thrce-layer MLP update with u(.) and v(-) transforms, and applics the learned scaling to the skip path. The main output head produces the temperature field á(x,, y), while an auxiliary head estimates the spatial diffusivity map {(x,y) jointly recovering both solution and physical parameter.
PINNsFormer architecture: To handle two-dimensional heat. flow, PINNsformer is simplified single trans- former layer of dimension 32 with one attention head. The spatial coordinates (z, y) are mapped via learned 2D sinusoidal embeddings before entering a WaveAct-activated feed-forward block. Residual connections and normalization persist in this minimal transformer. Its two output heads predict the temperature field ú(v, y) and the diffusivity field A(ι, y), embedding both solution and paramcter estimation within a physics-aware attention framework.
The models are trained on a grid-sampled interval with a batch size of 2500 for 1000 epochs. The initial learning rate is set 10-3, with Adam optimizer paramcters 81 = 0.9, Bg 0.999.
The results are quantitatively summarized in table 2 and the plots are Fig. 18-20.
B.3
Reaction Diffusion equation
B.3.1
Reaction-Diffssion Model for Chromatin Organization
The spatial organization of chromatin within the nucleus plays a fundamental role in gene regulation, in- fluencing cellular function and fate (Gilbert et al. (2005)). Chromatin exists in two primary states: hete- rochromatin, which compacted and transcriptionally inactive, and euchromatin, which more open and transcriptionally active. The dynamic interplay between these states is governed by diffusion and biochem- ical reactions, particularly modifications such as methylation and acctylation KKnn et al. (022)): Vinayak et al. (025
24

---

Published in Transactions on Machine Learning Research (MM/YYY)
Mathematical Model:
To model this system, the authors Kann et al. (2024) consider the mucleus as mixture of three components: nucleoplasm (øn), euchromatin (0e), and heterochromatin (Φn), where their volume fractions satisfy the space-filing constraint:
(n(x,1) + (n, ()(x,))= =1.
(47)
To simplify the system, they define two independent variables:
Nucleoplasm volume fraction (#, ()), Represents the proportion nucleoplasm at a given loca- tion.
Chromatin phase difference (da(s, 1)): Defined as the difference between helerochromatin and eu- chromatin fractions:
da(x,t) = (2, (z(
(48)
The order parameter Φd(z, t) indicates whether region is euchromatin-rich (øa < 0) or heterochromatin-rich (oa > 0).
Free Energy Formulation: The free energy density F(x, t) the system is given by:
8² 5²
F(x, t) = [o + oo - +n) Von 2 + Vodl"
(49)
2
2
where:
The first term represents chromatin-chromatinn interactions, modeled using Flory-Hnggins-tppo potential with minima at Φn = (euchromatin) and Φ) =b (heterochromatinn)
The second term accounts for interfacial energy, penalizing sharp transitions between chromatin phases and promoting smooth boundaries. The parameter å controls the interface width.
Diffusion Kineties of Nucleoplasm: The diffusion of mucleoplasm (øn) is governed by the gradient of the chemical potential pn (n, (z,t), which drives the system toward a lower-energy configuration. The evolution of muclcoplasm follows the diffusion equation:
Oon pn:
(50)
Øt
where 2 is the Laplacian operator, ensuring conservative dynamics where total nucleoplasm content remains constant unless boundary conditions specify otherwise.
Reaction-Diffusion Model for Histone Modifications: Epigenetic modifications such as methylation and acetylation mediate the conversion between euchromatin and heterochromatin. The evolution of the chro- matin phase difference (éa) governed by:
aos V2pa + 2(x0. -bx),
(51)
Øt
where:
2 ua represents the diffusive spread of epigenetic modifications.
Aoe - models methylation-drive -riivnn conversion from euchromatin to heterochromatin.
The chemical potential μa associated with this transition is:
Nd - + thr. (amax (n) 2(h)) 82 d4-
(52)
25

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Thus, combining equation 51 and 52 yields system which is governed by fourth-order differential equation, indicating complex spatiotemporal dynamics.
Numerical Solution and Simulations: To numerically solve the system, we discretize the evolution equations for On and Φd. along with their corresponding chemical potentials am and pd. The simulations assume no net exchange of chromatin or nucleoplasm with the surroundings, though boundary conditions can be adjusted to model specific biological scenarios.
B.3.2 Training setup:
PINN architechure: The neural network comprises 10 layers, each with 250 nodes using tanh activation function. We use spatial Fourier features as positional encoding. Training is performed using the Adam optimizer, with a learning rate that progressively decreases during training.
PirateNet architechure:
PirateNet for the reaftion-diffusion system. For the reaction-diffusio PDE, PirateNet remains an cight-layer tanh MLP with 256-unit width. The inputs yr pass through the same 256-dimensional Fourier embedding before entering successive Modified Bottleneck blocks, each mixing residual updates via learnable u and functions and scaling skips by a. Two output heads then predict the concentration field Óa(x, y) and the reaction-rate field X(x, y), capturing both diffusion dynamics and reaction kinetics in one cohesive residual architecture.
PINNsFormer architechure: For 32 with a single attention head. Inputs (x, y) reccive learned simusoidal emboddings and pass through a WaveAct feed-forward subblock, all under full residual connections and layer normalization. The model's main head outputs the concentration field (a(a,, y); while an auxiliary head outputs the reaction-rate map A(x,y) y), integrating physics constraints directly into the attention based architecture.
PINNsformer again uses one transformer layer of size
The models were trained on a grid sampled on the interval using a batch size of 2500 for 500 epochs. A learning rate of 1 10-3 with cosine decay scheduling was used. Training employed the Adam optimizer with specific parameters: 31 0.9, $2 0.999.
B.4 Correlation calculations, computing time estimates and memory efficiency estimates
We assess the accuracy and computational efficiency of our model by measuring the correlation between the actual and predicted parameter values, laplacians and the time/memory usage respectively. These evaluations ensure that our method not only effective but also feasible for real-world applications where resources such as time and memory are constrained.
B.4.1
Mean and Noise Calculation
To evaluate the robustness of our model and assess its sensitivity to statistical fluctuations, conduct multiple runs on the same problem. Each run produces two key metries: the predicted mean and the predicted noise.
Predicted Mean: The model outputs an estimate the central tendency of the target variable for each run. This predicted mean represents the model's best approximation of the expected output given the input features. By averaging the predicted means across multiple runs, we obtain consolidated estimate of the model's central tendency.
Predicted variance: In addition to the predicted mean, we compute the variability noise associated with each prediction. By averaging these predicted noise values, we gain insights into the uncertainty of the model's predictions.
Correlation Analysis: Once the actual and predicted mean, variance and the laplacians are computed, the next step is assess the correlation between the actual and predicted values. This ensures that the model effectively captures underlying trends and relationships in the data.
26

---

Published in Transactions on Machine Learning Research (MM/YYY)
To quantify this relationship, compute the correlation coefficient, defined as:
a
-
(53)
(X, X)2 (Y, P)2
where:
X; and Y represent the actual and predicted values, respectively.
X and y denote the mean of the actual and predicted values.
This correlation coefficient provides quantitative mcasure of the strength and direction of the relationship between actual and predicted values.
B.4.2 Time Evaluation
To measure the execution time of our model, we use Python's built-in time module, which provides simple way to capture the start and end times of computation. By recording the time before and after running the model, we can compute the total time taken for a single run. This approach allows us to evaluate the speed of our model under various conditions and for different input sizes.
Additionally, execution time measurements across different experiments enable the identification of perfor- mance bottlenecks and assessment of whether the model's time complexity scales appropriately with increasing data model size.
B.4.3 Memory Evaluation
To measure memory consumption, we utilize PyTorch's built-in utilities for tracking memory us- age during training and inference. Specifically, when using GPUs, we employ the functions torch. cudd. memory_.lllooatte and torch. cuda. wwe.mmmmmrorrrreeeeee () to monitor memory allocation.
By tracking both GPU and system memory consumption, efficient and ensure that it operates within available system resources without causing memory overflow or excessive usage.
determine whether the model is memory-
B.5 Forward toy problem
While our focus is on inverse problems, as demonstrated extensively throughout this work, we briefly illustrate the applicability of mollifier layers to a forward problem. The forward problem governed by the linear ordinary differential equation _ (xx) + Au(a:) = O, with À 1.0. The solution u(x:) is approximated using fully connectod neural network with hidden layer of 64 neurons each and Tanh activations. To computc spatial derivatives in stable and differentiable manner, we employ mollifier-based convolutions. The loss function consists of two terms:
ODE loss that enforces the differential equation, and a data loss that aligns the mollified prediction with the sampled data. We use mollifier kernel of size 11 with smoothing radius e = 5Ar, where the spatial discretization Ar = 0.05. The model is trained using the Adam optimizer with cosine learning rate decay over 5000 epochs. This example illustrates how mollifier layers can be used in the forward problem. The predictions as well as the learning curves are provided in Fig. 26.
C Super-resolution image processing
Since the super-resolution images are binary, they only indicate the presence or absence of signal a given locus. The density of these activated loci reflects the chromatin density at a given position, where a higher density corresponds to more heterochromatin. To establish correspondence between the imaging
27

---

Published in Transactions on Machine Learning Research (MM/YYYY)
data and the reaction-diffusion system, it is necessary to determine density equivalent to the obtained super-resolution imaging data. In this work, we define new square grid to better identify the distribution of chromatin within the nucleus. To compute the density of chromatin at a given point on this grid, we use the following methodology:
Watson Kernel interpolation Watsson (1644) is employed to perform spatial interpolation of data points super-resolution images. This method utilizes a Gaussian kernel compute weighted average estimate of nearby points, ensuring smooth interpolation over the image grid. The Gaussian kernel is defined as:
da
exp
20
((d)
(54)
V2T22
where d represents the Euclidean distance between points, and is the kernel's bandwidth parameter, which controls the kernel width. The kernel assigns higher weights to closer points, effectively smoothing the interpolation based proximal density.
Then the Nadaraya- Watson estimator (adaraya (1964); Watson (1964)) is used to compute the weighted average of data values, where the weights are determined by the above Gaussian kernel. Specifically, for a query point (Fnew: Unew), the estimated value žnew calculated as:
W
new
(55)
W/
where wi the weight assigned to each data point based on its distance to (Inew know), and z; is the corresponding value at each data point.
For computational efficiency, batch processing is used to perform interpolation in chunks. This method divides the larget grid into smaller batches and computes the weighted average for each batch, ensuring that the interpolation process scales efficiently with large datasets. The interpolation is implemented using the following steps:
Compute the distances between target points and known data points.
Apply the Gaussian kernel compute weights.
3. Calculate the weighted sum of the data values.
4. Normalize the weighted sum by the total weights to obtain the interpolated values.
After applying Watson Kernel interpolation, the interpolated values are used to update image features such as Φh, de, and on. These terms are computed as:
Zmesh
6
0.5
Φh
(56)
7
Fmesh 0.7
6
On
0.2,
(57)
0.7
7
0.7
Φe Oh
On-
(58)
These processed values are then used for further analysis, providing refined estimates of the underlying super-resolution imaging features.
28

---

Published in Transactions on Machine Learning Research (MM/YYYY)
D Extension to Multiple Unknown Parameters / Fields
This appendix formalizes how our mollifier-layer framework extends from a single unknown paramcter field to the case of multiple unknown parameters (or multiple coupled fields). The key point is that mollifier layers define a derivative operator applied at the output, and thus extend component wise to any number of predicted quantities.
D.1 Multi-parameter inverse- PDE formulation
Consider a PDE constraint written abstractly as
F Fl1,,, u(t,x), ())((1)))a1ss s(1,2r) 0,
(59)
where : T Ω2 + Rd, the (possibly pector-valued, state, and X(t,
At, x)) e R P collects P unknown parameters paraeeterssfields /fillds to be inferred. The PDE residual typically requires derivatives of up to order k.
D.2 Network outputs and mollifier layer (component-wise)
In our method, the base model predicts pre-mollified output and the mollifier layer produces the smoothed field û by convolution with compactly supported mollifier ry:
a(n) (g + (n) = m Í(m) n)(n - m) dm,
(60)
and derivatives computed by convolving with analytic derivatives of m:
@,ū(n) = (9 (n))
d0 (g * (n),
(61)
as in Eag. (4-6) of the main paper.
For multiple unknown parameters, we predict them as additional outputs of the model. parameterization is
convenient
(1, - (0(t, \(t,,)),
(62)
where may be (i) predicted directly, or (ii) predicted via its own pre-mollified outputs and (optionally) mollified in the same way as if one desires the same smoothing annotting/reguulriiztioo regularization:
Å (n) (ox, ()),, p= l,..,, P (optional).
(63)
Importantly, these equations are applied component- wise when u vector-valued; no change is required beyond carrying extra output channels.
D.3 Residual and training objective
Using mollifier-based derivatives, we define the residual at collocation points (l,, v:):
21), = F (1,22 24), (0) a((),
(64)
Training then minimizes the same PhiML objective as the single-parameter case, with Â replacing Å=
No N. Σ [ā((t, - u(ta, 13
MSE, 1 N N (f(tt,,;))1
MSErotal = MSE, + MSE,, MSE,
Σ
(65)
If supervision or priors are available for any parameter fields, one can add additional terms (c.g., 1 - smoothness regularizers), but. these not required for the extension.
29

---

Published in Transactions on Machine Learning Research (MM/YYY)
D.4 Remarks
What changes with P> 1. Only the dimensionality of the parameter output increases from Å to A; mollifier derivative computation for unchanged.
When to mollify parameters. If the PDE uses derivatives of paramcters (or one wants additional amoothing/reguulriizn on Å), apply the same mollifier construction to Åo via Eq. equation 63 otherwise predicting Å directly is sufficient.
Separable/closed parameter recovery. For PDEs where parameters can be expressed algebraically from predicted quantities (as discussed in Sec. 3.3), the same idea applies component- wise to each parameter in À whenever such a separable form exists.
E
Code and data availability
Code and data generated: All the code generated, along with the required data for the paper has been stored here: attp:::/0o...rr/11..522///////
15420249
STORM imaging data availability: We have provided the data for the 5 sections for which the parameters have been calculated with the uploaded data. A full sample image can be found at the Dhankhar et al. 2025 author's github repository at: https://github com/ShenoyLab//TTRN Ana
30

---

Published in Transactions on Machine Learning Research (MM/YYY)
a)
PINN Result at constant parameter
Actual Moan VS Predicted Moan (No Spatial Variation)
Actual (-1) Vs Predicted parameter
1,0
Prodicted
Actuale
0.5
0.0
Predicted Mean
0.0
-0.5
-1.0
-1.5
-1.0
-1.
-20
-3.0
b)
Mollifier Result at constant parameter
Actual Mean VS Predicted Mean (No Spatial Variation)
Actual (-1) Vs Predicted parameter
1.0
Prodlcted
Sti Day
Prodicted
0.5
0.0
Predicted Mean
-0.5
-0.5
Hark ai
-1.
-1.5
-2.0
-2.0
-1.5
-1.0
-0.5
0.9
0.5
1.0
20
40
60
80
100
Actual Mean
Figure 5: Constant forcing term inference in Langevin equation using PINN and Mollified PINN
31

---

Published in Transactions on Machine Learning Research (MM/YYX)
a)
PirateNet Results with constant parameter
Actual Predicted Spatial Variation)
(-1) Predicted parameter
1.0
1,0
Predicted Mean
Actual
0.5
0.5
0.0
Producted
-0.5
-1.0
-1.5
-2.0
-2.
-30
-2.0
b)
Mollified PirateNet Results with constant parameter
Actual Predicted Spatial Variation)
Actual Predicted paramster
1.0
1.0
Predicted Mean
Actual
Fredicted
0.5
0.5
Predicted
-0.5
-1.0
-1.5
-2.0
-2.
-2.0
-1.5
-1.0
-0.5
0.0
40
60
80
100
Actual
Figure 6: Conslant, forcing term inference in Langevin equation using PirateNet and Mollified PiraleNet.
PINNsFormer results with constant parameter
Actual Mean VS Predicted Mean (No Spatial Variation)
Actual (-1) Vs Predicted parameter
Sul Dey
Prodict aed
0.0
0.0
-0.5
Prodicted
-0.5
-1.0
-1.0
-1.5
-2.5
-2.0
-3.0
Actual Moen
Figure 7: Constant forcing term inference using PINNsFormer
32

---

Published in Transactions on Machine Learning Research (MM/YYY)
a)
PINN Result
Actual( meon -1 noise 45 ) Vs Predicted parameter
Actual( mean -1 noise
Vs Predicted parameter
-2
-4
-6
-8
-0
Actuall Predicted
Actual
Predacted
-10
20
100
20
100
b)
Mollifier Net Result
Actual( mean -1 noise .45 ) Vs Predicted parameter
Actual( mean -1 noise
Vs Predicted parameter
0.5
Predicted
Predicted
0055
-1.0
-2.0
-2.5
43
40
Figure 8: Noisy forcing term inference in Langevin equation using PINN and Mollified PINN
33

---

Published in Transactions on Machine Learning Research (MM/YYY)
a)
PirateNet Result
Actuall mean -1 noise .45) Vs Predicted parameter
Actual( mean -1 noise
Vs Predicted parameter
600
Actual
Actual
Preducted
b)
Mollified PirateNet Result
Actual( mean -1 noise .45 Vs Predicted parameter
Actual( mean noise
Vs Predicted parameter
0.5
Prodicted
Predicted
0.0
40
B0
100
4.0
80
100
Figure 9: Noisy forcing term inference in Langevin equation using PirateNet and Mollified PirateNet
34

---

Published in Transactions on Machine Learning Research (MM/YYY)
PINNsFormer prediction with noisy constant parameter
Actual( noise .45 Vs Predicted parameter
Actual( noise
Vs Predicted parameter
Actual
Actual
Predicted
Predicted
0.0
000
-1.0
-2.0
60
80
100
Figure 10: Noisy foreing term inference in Langevin equation using PINNsFormer
35

---

Published in Transactions on Machine Learning Research (MM/YYN
a)
Mean Predicted trend with Kernel Type
crder poyyomiia
polymorziaa
1.0
1.0
1.0
0.4
Prodzee Stemn
0.0
Prodtred MAND
0.-
-1.
-15
2.0
-2.0
Astual Meae
Actral Moon
Actusl Meos
b)
Mean Predicted trend with Kernel Size
kernel size 7
kemel size 11
1.0
0.5
0.5
Predicted Mean
Prodicted Moan
-0.5
-0.5
-1.0
-1.5
-1.5
-2.0
-2.0
Actual Moan
Actual Moen
kernel 15
kemel
1.0
0.5
0.5
Predicted Mean
Prodicted Meau
0.0
-0.5
-0.5
-1.5
-1.5
-2.0
-1.5
-1.0
-0.5
1.0
-2.0
-1.5
-1.0
0.0
0.5
Actual Mean
Actual Meen
Figure 11: Mean Forcing term prediction with differrnt kernel choices with Mollified PINNs
36

---

Published in Transactions on Machine Learning Research
b
1000
1000
2004
100
190
6:0
934
+081 4.38
+180
180
033
1400
Noise Variation wrt to differet kernel sizes
Noise Variation wrt differet kernel types
2.00
polyzomial
kemel size 11
and polytomial
1.75
kernel size
1.30
Prodicted Notse
1.25
Predicted
3.00
0.75
0.50
0.25
0.00
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
0.00
6.25
0.30
0.75
1.00
1.25
1.50
1.75
2,00
Actual Noise
Actual Noise
Figure 12: Estimating the Noise in the forcing term prediction with different kernel choices with Mollified PINNs
37

---

Published in Transactions on Machine Learning Research
PINN Result on temporal trend
Actual Vs Predicted parameter
Actual Vs Predicted parameter
0.0
Prodicted
-0.2
-0.2
-0.4
A00
AC0
-0.8
-0.8
-1.0
Predicted
40
100
20
40
60
80
100
PINN Result on temporal trend with white Noise
Actual Vs Predicted parameter
Actual Vs Predicted paramoter
Predicted,
Prodicted
Figure 13: Estimating with and without noise temporally vorying forcing term with PINNs
38

---

Published in Transactions on Machine Learning Research (MM/YYY)
PirateNet Result on temporal trend
Actual Vs Predicted parameter
Actual Vs Predicted parameter
0.0
Predictadd c
-02
-0.2
3
-0
-0.6
-1.0
Prodicted
-1.0
PirateNet Result on temporal trend with noise
Actual Vs Predicted parameter
Actual Predicted parameter
-2
Actual
Actual
Prodicted
Dredicted
100
Figure 14: Estimating with and without noise temporally varying foreing term with PirateNet
39

---

Published in Transactions on Machine Learning Research
Mollified PirateNet on temporal trend
Actual Predicted parameter
Actual Predicted parameter
Actiall
0.0
-02
-0.2
-0.6
-0.5
0.8
-1.0
Actual
Prodicted
20
40
20
100
Mollified PirateNet on temporal trend with noise
Actual Predicted parameter
Actual Predicted parameter
-2-
Artual
Actual
Predicted
Predicted
20
60
100
40
00
100
Figure 15: Estimating with and without noise lemporally varying forcing term with Mollified PirateNel
40

---

Published in Transactions on Machine Learning Research
PINNsFormer results on temporal trend
Actual Vs Predicted parameter
Actual Vs Predicted parameter
0.0
0.0
Predictec.
-0.2
-0.6
-0.0
-0.6
-0.6
-0.0
-0.0
-1.0
Predicted
-1.0
20
40
BC
100
40
60
80
100
PINNs results on temporal trend with white noise
Actual Vs Predicted parameter
Actual Vs Predicted parameter
Actual Predicted
Actual
Predicted
Figure 16: Estimating with and without noise temporally varying forcing term with PINNsFormer
41

---

Published in Transactions on Machine Learning Research (M//YYY))
Constant Parameter
Spatially Varying Parameter
Spatially Varying Parameter
Parameter
10.00
Actual
12 U
Parameter
Predicted
PINN
12 u
Figure 17: Estimating constont and spotially varying diffusivity terms and corresponding Laplacians for the
heat equation using PINNs
Constant Parameter
Spatially Varing Parameter
Spatially Varing Parameter
12 Parameter
Actual
Pre
Parameter
Predicted
Figure 18: Estimating constant and spatially varying diffusivity terms and corresponding Loplacians for the heat equation using PirateNets
42

---

Published in Transactions on Machine Learning Research (MM/YYN)
Constant Parameter
Spatially Varing Parameter
Spatially Varing Parameter
Amarlooo
Parameter
Actual
Adaiti
Adtal 17
V2
Tridanad READ
Prdfeed th:0
Parameter
Predicted
Q2
Figure 19: Estimating constont and spatially varying diffusivity terms and corresponding Loplacions for the
heat equation using mollified PirateNets
Constant Parameter
Spatially Varing Parameter
Spatially Varing Parameter
Parameter
Actual
kndd
722 U
Parameter
Predicted
V2 U
Figure 20: Estimating constant and spalãally varying diffusiviit terms and corresponding Laplacians for the heat equation using PINNsFormer
43

---

Published in Transactions on Machine Learning Research (MM/YYY)
a)
b)
PINN Result
Mollifier Net Result
Actual Mean VS Predicted Mean (No Spatial Variation)
Actual Mean Predicted Spatial Variation)
Predicted Meen
Predicted Moan
Prodicted Mean
0.8
Pradicted Mean
1.0
06
0.8
0.4
0.2
0.4
0.6
0.8
1.0
1.2
Actual Mean
Actual Mean
Figure 21: Predcited us. Actual mean for the reaction rates in simulations for PINNs and Mollified PINN
PirateNet mean prediction
PirateNet+M mean prediction
Actual Mean VS Predicted Mean (No Spatial Varialion)
Actual Mean VS Predicted Mean (No Spatial Variation)
Predicted Moun
Predicted
Dev
Sid Dev
1.2
Predicted Moan
1.0
Producted
0.6
0.4
-0.2
Actual Mean
Actual Mean
Figure 22: Predeiled ws. PirateNet
Actual mean for the reaction rales in simulations for PirateNet and Mollified
44

---

Published in Transactions on Machine Learning Research
PirateNet noise prediction
PirateNet+M noise prediction
Prodicted
3.200
Prodicted
Stl Dev
Std Dev
0.7
3.150
Frediceed Noise
Prodicted
0.100
3.075
0.4
3.025
0.4
0.6
0.8
1.0
Actual Notse
Actual Noise
Figure 23: Predciled us. Actual noise for the reaction roles in simulations for PiraleNet and Mollified PirateNet
Mean and noise predictions for PINNsFormer
Actual Mean Predicted Mean
Spatial Variation)
Actual Mean VS Predicted Mean (With Spatial Variation)
Predicted Moon
Predicted Nouse
Std Dos
1.2
0.7
1.1
Predicted Mean
1.0
0.5
0.8
0.7
0.3
0.5
0.1
Actual Mean
Actual. Noise
Figure 24: Predicled vs. Actual mean and noise for the reaction rales in simalations for PINNsPormer
45

---

Published in Transactions on Machine Learning Research (MM/YYN
Ground truth Pa
MSE loss
PDE loss
10
25
50
101
10a4
75
Loss
1.005
02
100
0.1
150
10
1019
50
100
150
200
0
230
500
750 Epochs
1000
1250
1500
0
250
500
750
1000
1250
1500
X
Epochs
Predicted S
Predicted Laplacian
0
555
0.0080
2.5
50
75
0.0125
52
100
125
5.1
125
GLI075
150
150
0.009
175
-03
200
200
50
100
150
200
0
50
100
150
200
X
Figure 25: Mollified PINNs capture the spatial reaction rates for difftion equation.
Prediction of u(t)
Training MSE Loss
Training ODE Loss
Ground Mollifer
162
000 Toos
10:
Rosidual
107
0.4
10
10
MSI
0.0
10:3
kpoch
Egoch
Figure 26: Using Mollified PINNs for a forward problem.
46


Reviews:
[{"review_id": "JN73AZVCVb", "reviewer": "Reviewer_vw5y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper address a common scientific challenge related to parameter estimation in inverse problems involving partial differential equations where unknown model parameters may vary in space or time and is to be estimated given sparsely sampled data.\n\nThe authors proposes to deal with high-order derivatives in physics-informed machine learning approaches with mollifier layers instead of relying on automatic differentiation with the aim to reduce the overall computational costs and improve the performance through more accurate estimation of derivative computations. By introducing mollifier layers it is possible to reframe the derivative computation as a smoothening integration that may be done by attaching mollifier layers to the output layer without any other architectural modifications. Comparisons of this proposed network design technique is done against a  physics-informed neural network (PINN) approach for different inverse PDE problems that are representative for commonly encountered problems in science and including a real-world application of relevance in biomedical areas. \n\nThe new mollifier layer technique is aimed at lowering the computational and memory costs of dealing with high-order derivatives and also give support to improved accuracy for modeling spate-temporal varying systems and systems with noisy parameters. The proposed technique is interesting as it is a new/alternative way to use and possibly improve physics-informed neural networks for a class of problems.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Physics-informed machine learning techniques have seen a revival in recent years and given strong attention in the research communities. The proposed mollifier techniques is aimed at improving the naive PINN approach and with emphasis on parameters estimation done when solve inverse PDEs, which constitute an important set of scientific problems that span science and engineering.\n\nThe mollifier techniques are well-known are important, and it is interesting to see it being used to improve PINNs for parameter estimation when solving inverse PDEs. This is a new idea to the best of my knowledge and for this reason the study is highly relevant.\n\nThe benchmark PDE problems considered, highlights that mollifier layers can improve PINN-type approaches (cf. their Table 2), and in particular when performing parameters estimation with noisy data as a result of the smoothening property inherited from the mollifiers. In particular it is demonstrated that mollifier layers improve techniques such as PINN (with techniques laid out in section A.4) and PirateNet, and is better than the finite-difference based PINN approach where finite difference are sensitive to noisy data and hence perform poorly. The problems considered incudes arguably non-trivial time-varying parameters and spatially varying parameters, and problems of PDEs. How would the model work for problems with multiple parameters that is to determined simultaneously? ... this is not considered.\n\nA short coming of the paper is the lack of mentioning of other (competitive) techniques used for parameter estimation that is not based on neural networks and possibly including a comparison with such techniques. For example, there is the classical newton-based techniques such as framed in ODIL [1], e.g. see references given here. The authors of [1] challenges current PINN paradigm(!). [2] proposed recently using the closed-form least square solution procedure for ODEs (main emphasis) and PDEs (one example) and compared to (naive) PINNs and outperform also PINN in parameter estimation. \n \n[1] Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks \nPetr Karnakov , Sergey Litvinov , Petros Koumoutsakos Author Notes\nPNAS Nexus, Volume 3, Issue 1, January 2024, pgae005, https://doi.org/10.1093/pnasnexus/pgae005\nURL: https://academic.oup.com/pnasnexus/article/3/1/pgae005/7516080\n\n[2] Physics-Informed Regression: Parameter Estimation in Parameter-Linear Nonlinear Dynamic Models\nJonas Søeborg Nielsen, Marcus Galea Jacobsen, Albert Brincker Olson, Mads Peter Sørensen, Allan Peter Engsig-Karup, arXiv, 2025.\nhttps://arxiv.org/abs/2508.19249", "audience_yn": "Yes", "audience_explanation": "Yes the TMLR serve a technical community and the proposed mollifier technique is an interesting new idea that warrants evaluation.", "requested_changes": "It is suggested to expand the literature review and mention other state-of-the-art approached, e.g. ODIL, and discuss the relation to this work. See other comments given about shortcomings.\n\nIt is suggested to compare to non-neural network based techniques in the benchmarks.", "broader_impact_concerns": "No suggestion.", "additional_comments": "Overall well-written paper and interesting and relevant use of mollifiers with physics-informed machine learning.", "extra_scores": {}, "extra_text": {}}, {"review_id": "zIO9yV50Qe", "reviewer": "Reviewer_1F51", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**  \nThe paper introduces a mollifier layer, an architecture-agnostic extension to the standard network used in physics-informed machine learning (PhiML). The mollifier layer serves as a mechanism to compute efficient and noise-robust higher-order derivatives and serves as a replacement for the standard autodiff algorithm (which computes the derivatives but is both compute and memory inefficient and lacks robustness to noise).\n\n**Baselines**  \nSufficient baselines, showcasing the agnostic nature of the proposed method. Three PhiML architectures were reported. For each architecture, a comparison was made between the standard architecture (i.e., with autodiff) and its mollifier-augmented variant.\n\n**Reproducibility**  \nThe source code for the approach was published, which supports the reproducibility of the results and further research in the area.\n\n**Limitations**  \n1. The approach may underperform in computing the [higher order] derivatives when the data contains high frequency variation due to smoothness bias induced via the mollifier layer. As a mitigation strategy, the paper introduces an alternative estimation mechanism that is only applicable when $\\lambda$ is separable in the PDE. \n2. The approach currently requires manual fine-tuning to obtain the correct kernel size, and the choice of kernel is also very important. The approach is very sensitive to the frequency distribution of the underlying function.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims were validated by the empirical results across three off-the-shelf PhiML baselines. For each baseline, the mollified-augmented version showed better robustness to noise than the standard baseline (without the mollifier layer).", "audience_yn": "Yes", "audience_explanation": "The paper addresses a relevant problem in the sub-field of Physics Informed Machine Learning (PhiML), a growing sub-area of machine learning research. Furthermore, there are PhiML researchers within the TMLR community, as evidenced by previously accepted publications in the research area.", "requested_changes": "1. Typographical error in page 4, paragraph 2. “Here, $N_u$ are the training data points available for the actual solution $u(t, x)$ and $N_f$ *all* all the points in”. The first “all” should be “are”", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "LH6ebHtDlq", "reviewer": "Reviewer_osLu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript proposes mollification as a replacement of autodiff for computation of derivatives with respect to time and space in PINN-style models. \n\nIt provides evidence that mollification leads to lesser use of memory, faster training time, better mean, temporal and spatial correlation, as well as better Laplacian correlation. It also shows consistent performance for different orders of the underlying problems, unlike (and unsurprisingly) FD. \n\nAlthough I liked the idea described in the paper, I found sections 3 and 4 (and linked 5) to be problematic. \n\nThe key issue with section 3 is incompleteness. After reading that I had only a vague understanding of how to apply mollification in practice. The theoretical part occupies most of the section but largely focuses on nuances of when mollification is applicable. Even then I learnt more about that from reading a dedicated paper on the topic rather than the manuscript. I was hoping that after presenting eq 4, 5 and 6 (with second rather than third derivative), showing some examples of mollifying functions and their analytic derivatives, you would go to describing how these equations are integrated in Figure 1 (implementation through convolution with different kernel sizes).\n\nThe key issue with section 4 is again incompleteness. You chose a number of tasks to present but did so in a rather brief fashion redirecting the reader to your appendix on numerous occasions. I have found this to be a very distracting experience. I would have preferred to see one experiment to be fully described in the main body of the paper with others to be placed in the appendix.", "claims_and_evidence_yn": "No", "claims_explanation": "In my opinion table 2 provides good evidence that mollification works as expected. However, as mentioned above, I believe the manuscript\n\n1) could have been more clear about the technical details behind mollification;\n2) could have provided a more comprehensive description of each experiment chosen to go into the main body of the paper. \n\nAs such I cannot positively answer to this question.", "audience_yn": "Yes", "audience_explanation": "As I mentioned above, I liked the idea presented in this manuscript and the results shown also suggest this to be a good approach to capturing higher order derivatives more accurately than what is possible with autodiff.", "requested_changes": "1) Improve integration of theory and practice by providing more details how mollification is implemented, what mollifying functions were explored (not in appendix), what their derivatives are, how convolutions were implemented. \n\n2) Focus experimental part on one, maximum two experiments. For instance, the 4th order equation is a good choice. Please describe the experimental setup and experiments performed carefully and clearly. \n\n3) Please provide some technical/mathematical insight into poor performance of autodiff and compare that to mollifying. \n\n4) Switch from 3rd order to 2nd order derivative in eq 6.\n\n5) Please ensure equal ranges of x and y axis on any plot of predicted vs actual noise. I would like to see (non)-diagonality easier without the need to check ranges every single time. \n\n6) Please explain every single symbol used in the main body in the main body (e.g. \\Lambda, \\sigma on page 7). \n\n7) Please explain a rather weaker performance of mollifying in PDE loss minimisation in Fig 2c including what it implies. \n\n8) Please explain why \\phi_n is missing in section 4.3 after its first mention. \n\n9) Please explain/more closely link the text in section 5 to figure 4.\n\n10) Please double check all of your figures in the appendix. For example, Figure 11 shows identical mean prediction performance, which does not make sense. \n\n11) Please add space between text and citation everywhere in the manuscript (e.g. noisy measurements(Kaipio... -> noisy measurements (Kaipio...)", "broader_impact_concerns": "No concerns.", "additional_comments": "No additional comments.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)